Website requirements

[chrisdickinson]

Replacing #27 in order to pin this. I want give more context on the pending software design decisions we need to make in order to build out a website (this affects #102 and #103, which are much appreciated!)

the problem

Entropic needs a website, because:

1. In order to allow people to sign up for accounts, we use GitHub OAuth login for signup and token creation
2. In order to allow people to manage their account, we display a token management UI.
3. In order to aid discovery & vetting of packages, we want to display information about packages (dependencies, dependents, etc.)
4. In order to aid vetting of packages, we want to display information about package contents (styled, rendered package contents as html)

The current website doesn't have any styles, is likely inaccessible, and is hard for developers to extend. Importantly, it's also embedded into the registry service, which could present security & caching concerns down the line.

constraints, rakes in the yard, & open questions

splitting into services

At present, Entropic runs in a single process. Because certain tasks (like syncing external packages) are too time-consuming to run in the duration of a request/response cycle, we need to split this out into multiple services. Eventually, I believe we'll need 4 services:

1. The existing external registry API service we expose today, talking to...
2. A ✨new✨ internal api encapsulating our data stores that the website, worker, and registry service can leverage.
3. A ✨new✨ external website service talking to that same internal api
4. A ✨new✨ worker service for processing syncing requests, talking to the internal api

For the production entropic instance, @ceejbot took a clever approach with nginx proxy_pass's so that we can simulate privileged APIs for the website to use by mounting them on the registry at non-exposed routes (for example, /private/.) This lets us start working on the website without having completed the other two ✨new✨ apis. However, until we split out the website process from the registry process, web handlers will use HTTP clients to talk to internal handlers in the same process. We'll start this process for the registry as well.

This is quite a mouthful: this is all to say that when the website starts needing to get at registry data, please pull in @chrisdickinson and/or @ceejbot.

Single page app or Server Side Template Language?

We'll want to consider whether we use a server-side templating language like Nunjucks to render pages in a traditional fashion, or go a single page app route using something like React + Next.js. Right now I am leaning towards server side templating using Nunjucks (or similar):

• Our present Oauth & CLI signup needs lean on url routing. (They could, yanno, not. But.)
• Our form needs are pretty basic.
• SPA's introduce a certain level of maintenance complexity (bundling FE assets, etc.) Next solves this pretty nicely, which is one of the (many) things I like about it, but it's still a cost.

I could be wrong here, so please poke holes in this. What are we buying with a single page app?

CSS

Let us do the simplest thing that works, then iterate. I propose we use Tachyons for a base coat, then specialize with an added stylesheet. (When this gets unwieldy, that's when we should start looking into css-modules or css-in-js as an additional layer on top of tachyons.)

[superhawk610]

In my experience, any project that requires more than the most simple frontend will see large DX benefits when using an SPA over simpler template rendering. If the frontend were only required for signup/login/auth, I think an argument could be made in favor of a templating language. However, since it also aims to create package search/detail/statistics interfaces, I believe the SPA approach becomes more appealing.

I foresee the following elements requiring a non-negligible amount of client-side scripting:

• search autocompletion
• package description markdown rendering
• interactive UI (tabs, form autocomplete/validation, routing, etc)

If those features aren't necessary and can be avoided, then an SPA may be overkill. However, as soon as you want to work on one of them, you'll have to write client-side JS, which will lead to some sort of client-side dependency bundling/processing to optimize in production, so I believe that maintenance complexity will naturally arise one way or another - may as well take advantage of the DX boosts associated with an SPA:

• better separation of concerns (you can't tightly couple your server logic and frontend when they're split into two separate projects)
• hot code reloading for faster design iteration
• better end-user experience (only a single initial page load, no page flashes when changing routes)
• easy access to more complex build tools (static typing, CSS-in-JS, minification, asset optimization, etc)
• the web client can make use of the same REST routes that the CLI client uses, so more streamlined experience between the two
• assets can be cached client-side and don't need to be regenerated on every request like with a templating engine

The biggest downside of an SPA is definitely the added complexity, but I personally believe that cost is worthwhile and will lead to a better product long-term. I'm interested to see what others have to say - I'll press the pause button on #102 until this discussion progresses further.

[tunnckoCore]

@superhawk610 absolutely agree. SPAs may sound complex, but for me, they aren't and definitely worth all the features, pluses, minuses, and optimizations that they give - not only DX is bumped to the skies but also the UX.

Just to throw it here: Bet all-in on Gatsby, except maybe for the "internal" one which makes sense to be Next.js

[jakeburden]

Don't know if you'd want to lean on or trust Google to keep the service up, but firebase offers a nice way to add Github OAuth to your site without needing to create url routes.

This is my typical setup for it:

// helpers/firebase.js
import * as firebase from 'firebase/app'

import 'firebase/auth'
import 'firebase/firestore'

import config from '../config/firebase'

const provider = new firebase.auth.GoogleAuthProvider()

const app = !firebase.apps.length
  ? firebase.initializeApp(config)
  : firebase.app()

const db = firebase.firestore()

export { app, provider, db }

// pages/index.js

import { app } from '../helpers/firebase'
import { useAuthState } from 'react-firebase-hooks/auth'

const Page = () => {
  const [user, initialising, error] = useAuthState(app.auth())

  return (
    <Layout>
      <NavWithGithubAuthButton user={user} />
      {user ? <p>Signed In!</p> : <p>Not!</p>}
    </Layout>
  )
}

export default Page

[superhawk610]

Firebase auth is probably overkill since entropic isn't using any other firebase features like firestore (everything's already set up to use self-hosted Redis/Postgres instances). The server-side logic for handling registration/login is already there, we just need a way to:

• receive and store an OAuth token client-side (replacing /www/login/providers/:provider/callback with a client-side route), or
• generate a corresponding entropic frontend token for the client to store that doesn't reveal the GitHub (or whatever provider) actual OAuth token and use that to authenticate any elevated request from the frontend, or
• make use of the existing session middleware and keep the OAuth implementation completely server-side (using cookies instead of Authentication: Bearer XXXX headers for elevated requests)

[Wizyma]

Hi @chrisdickinson i would love to work on this part and contribute to the project !
And i most definetly agree with @superhawk610 !

[trodrigues]

I was kind of anticipating this whole discussion about what technologies to use and not knowing what was necessary right now that’s why my PR (#103) was really just about the most simple HTML/CSS site possible right now so that at least some information about the project could exist in a website.

Being a frontend developer myself and having worked with all the popular frameworks, in large and small apps, client side only SPAs and server rendered SPAs and having managed all sorts of complexity, I’ve also went with a super simple approach for my own personal website (using eleventy+nunjucks) because I really didn’t need more, and I think a similar approach could work here, similar to what Chris described:

• Start small, build just some static pages
• Using a templating system like nunjucks (which is mostly some templating to display data and some conditionals and loops) doesn’t lock you in too much
• First bits of javascript based on the needs described above are probably small enough that they don’t even need to be bundled
• If you do need to start bundling, you can do so with the most basic webpack/parcel build, just to put some files together
• If there’s a need to go the SPA (or SSR SPA) route after that there shouldn’t be too much lock in and it should be straightforward to migrate

TL;DR: start with something simple, add tooling as necessary.

[superhawk610]

First bits of javascript based on the needs described above are probably small enough that they don’t even need to be bundled

I don't think that's true since we'll need a client-side markdown rendering library (and whatever dependencies it brings with it). Granted, we could render/cache project READMEs server-side, but then that introduces a non-negligible overhead at scale and I feel like one of the benefits of this project is that it's usable by anyone, no need to drive up the hosting costs. Same for server-side rendering, at scale it will require higher container compute time and will more quickly require higher hosting tiers, while an SPA can be thrown behind a Cloudflare proxy and forgotten about.

If there’s a need to go the SPA (or SSR SPA) route after that there shouldn’t be too much lock in and it should be straightforward to migrate

I think this project is straightforward enough to know now whether or not this will ever be required; the project's scope is pretty well defined, and there are existing alternatives like NPM to use as a case study (NPM does use an SPA). While I appreciate the desire to start small and work your way up, why not go ahead and plan for the future now? If we go with templated SSR now and then 6 months or a year later want to switch, all of the template code will have been wasted since it won't be able to translate to SPA land.

Additionally, SPAs will allow for a more useful PR diffing experience since changes to the frontend UI will never touch the backend files, and vice versa. It will be clear from filename alone whether changes were made to the visual presentation or to the business logic, and contributors that only want to participate in one won't have to slog through the other.

From the points already provided I think it's fair to say that:

• SSR templating => easier maintainability, worse user experience
• SPA => tougher maintainability, better user experience

In my mind, better UX should always win out.

[jefflembeck]

NPM does use an SPA

npm uses a SPA but it’s barely necessary and was more of a way for us to dog food common ways for our users to build a web application. (Also the entire site is SSR + rehydrated front end)

[zacanger]

FWIW (coming from a recovering FED), SSR with only minimal client-side JS would make things a lot easier for folks who like to browse with JS off or use alternative browsers (including text-mode browsers). And you could get a lot of the DX benefits with SSR by using something like Next.

[wardpeet]

I'm biased in favor of gatsby but gatsby will be great for this. You don't have to run a server because everything is static hosted so it's going to be pretty cheap. Api calls can be cloud functions as they can scale pretty heavily.

For the packages pages itself it's probably to best to go for full client side for now as gatsby doesn't have incremental build support yet which means we have to rebuild all pages.

I don't mind guiding/writing a bunch of the website in gatsby if that's the direction you want to go.

[iainjreid]

From the points already provided I think it's fair to say that:

• SSR templating => easier maintainability, worse user experience
• SPA => tougher maintainability, better user experience

Just my two-penny worth, but I would also lean towards a more user-centric approach. This project will likely appeal to a wide range of frontend developers and designers alike, and the appearance and feel of tools can impact heavily on how or even if they are adopted.

[venikx]

I'd like to raise a few questions before jumping into the SPA vs SSR debate:

• How do SPA's deal with search engine indexing? Is it a goal to optimize it to "compete" with npm?
• How do SPA's deal with minimal browsers?
• What is defined "good UX"? What is considered the main use-case of the registry?
  It's important to ask, because depending on the answer SSR or SPA can be the better user experience (imho). I don't mind that an mail application client takes a while to load in the browser, because I might "live" in it for a while, once it's loaded. I do mind slow loading speeds when browsing content (let's say Medium). So it's important to understand where entropic would fall onto this spectrum.
• Is DX valued more or less, compared to UX? And how bad/good are both sides?
• What is more valuable when thinking about server-side (cost) vs client-side time (perf)?

[nilsnh]

You raise good questions @venikx.

I'd like to raise a few questions before jumping into the SPA vs SSR debate:

* How do SPA's deal with search engine indexing? Is it a goal to optimize it to "compete" with npm?

According to Google they seem to be able to index SPAs decently since 2015. But Facebook doesn't parse JS (Could that be the same case for Twitter?).

* How do SPA's deal with minimal browsers?

SPA won't work in text mode browsers (I think?). If we're transpiling javascript we'll be able to support old browsers.

* What is defined "good UX"? What is considered the main use-case of the registry?

One way to define good user experience is that the site is as accessible as possible to the diverse Javascript ecosystem. Like, it should definitely be accessible by screen readers. It should be easy for newcomers to learn what it is and isn't. Likewise, it should be an effective site for more experienced users.

  It's important to ask, because depending on the answer SSR or SPA can be the better user experience (imho). I don't mind that an mail application client takes a while to load in the browser, because I might "live" in it for a while, once it's loaded. I do mind slow loading speeds when browsing content (let's say Medium). So it's important to understand where entropic would fall onto this spectrum.

* Is DX valued more or less, compared to UX? And how bad/good are both sides?

* What is more valuable when thinking about server-side (cost) vs client-side time (perf)?

All of this is just my five cents. 😸 Whatever we build should be maintainable for the future and as accessible as possible for screen readers, for newcomers and for experienced users. And we also need to consider localization. 💭

[alexgarces]

I agree with @trodrigues' approach. I think using SPA at this point is overkill. SPA is not a good solution for all use cases.

If we barely know the features we will need, why should the project be locked to SPA? This is anticipating the future and we are in a very early state.

SPA's are a great solution for high interactive web apps (like dashboards, for example). But SPA's are terrible for some very basic web features. For example, forget about decent indexing and SEO on SPA, compared to plain HTML or SSR.

Also, as @nilsnh suggests, do we want the website to be accesible for everyone or only for last-generation browsers? SPA's rely totally on JS which can easily end up with performance issues (so more complexity to deal with).

@chrisdickinson +1 to Tachyons!

[venikx]

@alexgarces I agree.
I'm a fan of SPA's, but that's because I mostly work on dashboards and I think that's an amazing case for SPA's, since there is a lot of user interaction.

If we really don't know the amount of features yet, I'd also opt for @trodrigues' approach. Start small, introduce more complexity when the benefits outweighs the trade-offs.

[superhawk610]

If we barely know the features we will need...we are in a very early state.

Yes and no - the project is young, but it is aiming to directly replace an existing product with years' of trial and error backing it. Entropic isn't aiming to reinvent the wheel, just to take an existing wheel (NPM) and change the way it is hosted and managed on a high level (at least, that's my understanding, please correct me if I'm wrong). As I listed earlier, I believe the featureset is already fairly well-defined:

• client-side markdown renderer
• client side form interactions (regex validation, checking for duplicate username before submissions, etc)
• client side UI interactions (tabbed panes, route transitions, loading states, etc)
• client-side asset caching

forget about decent indexing and SEO on SPA, compared to plain HTML or SSR.

SPA and SSR are not mutually exclusive. The only functionality exclusively available to an SSR is page-specific <meta> tags, and this can still be very easily accomplished by spinning up a very minimal SSR renderer (either using an existing solution like Gatsby/Next or even a simple Express server) to inject the relevant SEO info into the index.html before serving - this approach has the advantage of minimizing server-side work while still allowing for the richer UX of an SPA.

EDIT: I've added a PoC to my PR branch, you can check it out here.

Also, as @nilsnh suggests, do we want the website to be accesible for everyone or only for last-generation browsers? SPA's rely totally on JS which can easily end up with performance issues (so more complexity to deal with).

While they do rely on JS, SPAs also make quickly implementing and iterating on things like i18n and a11y much easier.

If we really don't know the amount of features yet

I've seen a few comments mention this line of logic, and I'm curious to hear more about your thought process - what aspects do you think are still left to decide on?

[chrisdickinson]

The feature set is not defined (that's on me, sorry!) This is not an exhaustive list & I will do a more complete writeup elsewhere as soon as I get some time!

• The hierarchy of values for the website are: we want the server to be easy to set up and maintain, easy to use, and easy to develop, in that order.
  • By "maintain", I mean: imagine someone outside the JS community being asked to set up an Entropic instance, or to fix an Entropic that's throwing errors.
• Entropic instances may live inside your network, so network latency concerns are a bit different than the average website.
• Markdown is pre-rendered on publish. It is stored at rest as HTML suitable for inlining. (Storage is cheaper than CPU cycles.)
• We don't have search. Federated search is going to be a different beast than centralized search. (Think Napster/Kazaa/Limewire/Bittorrent, not Google.) It might not be a thing that individual Entropic instances support!
• We want i18n support. My experience with rich client apps has been that you end up having to generate multiple different asset bundles for each target locale (or else do some kind of code-splitting.)
• We want to have good a11y.
• The server component of the website will have privileged access to an internal backing storage API that frontend code will not.
• We currently only supply the following functionality:
  • receive CLI login session
  • Redirect to GitHub OAuth
  • Receive GitHub OAuth response
  • Sign up for new user
  • Display token list
  • Create new token
  • Homepage (currently served by nginx as flat html)
• We wish to supply, roughly:
  • A paginated list of all packages on the instance
  • Package detail pages with:
    • A readme
    • links to dependencies,
    • links to (local) dependent packages.
    • No download counts (Content addressable store precludes this.)
    • No external dependent packages (we don't know who depends on this package elsewhere!)
  • List of namespaces of which you are a member.
  • List of outstanding namespace membership invites for your user.
  • For each namespace of which you are a member, a settings page:
    • List of extant maintainership invites.
    • (Long term) Ability to invite a member to a namespace.
    • List of current members.
    • List of packages maintained by this namespace.
      • (Long term) ability to invite a maintainer to a package.
      • (Long term) ability to remove a maintainer from a package
      • (Long term) ability to "yank" a package or package version
    • List of packages originally published by this namespace.
      • (Long term) ability to remove a maintainer from a package
  • (Long term) Create a new namespace

[superhawk610]

Markdown is pre-rendered on publish. It is stored at rest as HTML suitable for inlining. (Storage is cheaper than CPU cycles.)

Client side rendering is cheaper than both ;)

[wardpeet]

I don't think it needs to be a debate of ssr vs csr. Rather think about maintenance and community. When I said gatsby, I still think it's a good approach as it can handle CSR and prebuild ssr. We also focus on a11y and speed. The downside is that people need to "learn" react & graphql.

you'll have to think about:

The hierarchy of values for the website are: we want the server to be easy to set up and maintain, easy to use, and easy to develop, in that order.

By "maintain", I mean: imagine someone outside the JS community being asked to set up an Entropic instance, or to fix an Entropic that's throwing errors.
Entropic instances may live inside your network, so network latency concerns are a bit different than the average website.

[nilsnh]

Thank you for enumerating the requirements @chrisdickinson. :)

The downside is that people need to "learn" react & graphql.

I agree on your main points @wardpeet. :) I just want to note that even React and GraphQL will provide some barriers to entry, especially if we add their conventions of usage on top of that. React or GraphQL could definitely be the right tools here, but people will have to learn them, not "learn".

So, should we try to arrive at some common consensus, or create some minimal Proof of Concept stacks and argue for them? 🤷‍♀

[trygve-lie]

While they do rely on JS, SPAs also make quickly implementing and iterating on things like i18n and a11y much easier.

Thats not a given. That all boils down to how a project is structured and I sure have encountered SPAs which is close to impossible to develop on due to its complexity and poor structure.

SSR templating => easier maintainability, worse user experience
SPA => tougher maintainability, better user experience

These are not given facts either. There are tons of SPAs out there with horrible UX.

I would start out with a very basic SSR template based site and enrich with client side scripting for enhanced features and go from there. I would also add a service worker to cache all html which are fairly static and assets to reudce amount of hits to the server for these.

On a bit longer term I would honestly like to see usage of web components for the different features. It would be very nice if one where able to make ex search a reusable web component one used on the web site here but which also others could use to plug into their Vue, React, Svelte, etc app. From a project like this I would very much like to see reusable components which can be plugged into different frontend frameworks rather than going all in on Vue or React component.

[lacaulac]

I'm quite a fan of SPA so I'd naturally lean towards making one, but I really see two issues with what @chrisdickinson described if we were to use one: The maintainability for people setting up their own instance and the "private" API part that only the server code should be allowed to access, even though I don't think the latter should be too much of a problem. But again, the big pro of SPA is minimising the amount of work the server has to do by almost limiting it to API calls with the consequence of potentially breaking non-Javascript browsers. I think that starting to plan how we want the website to work now is a good approach, since being offered the choice of either tough maintenance or almost completely rewriting the website isn't that great.

[alexgarces]

@lacaulac I'm also a big fan of SPA's! The developer experience when working with React and similar technologies is awesome. But being part myself of a failed SPA project some months ago made me much more critical about the SPA hype. Since then, I try to be more practical and minimalistic. And the only use cases I find for SPA's are dashboards and high interactive apps.

@chrisdickinson thanks for listing the requirements!

By "maintain", I mean: imagine someone outside the JS community being asked to set up an Entropic instance, or to fix an Entropic that's throwing errors.

This is a key reason to avoid using a big framework for now.

Markdown is pre-rendered on publish. It is stored at rest as HTML suitable for inlining. (Storage is cheaper than CPU cycles.)

I'm going to contradict myself but, after reading this requirement, I would also consider Next.js. I've worked with Next.js and it is great for this particular use case. You can switch between SSR and CSR, and export it as a static site. The downside about using it, again: you are locked to the complexity of React from the beginning.

@superhawk610

SPA and SSR are not mutually exclusive. The only functionality exclusively available to an SSR is page-specific tags, and this can still be very easily accomplished by spinning up a very minimal SSR renderer (either using an existing solution like Gatsby/Next or even a simple Express server) to inject the relevant SEO info into the index.html before serving - this approach has the advantage of minimizing server-side work while still allowing for the richer UX of an SPA.

You are totally right, SPA and SSR are not mutually exlusive. But why adding the complexity of SPA before you need it?

SEO is not only about meta tags. Content is the most important thing for SEO. And client side rendered content via JS is very very far from having a decent indexing.

@trygve-lie

These are not given facts either. There are tons of SPAs out there with horrible UX. I would start out with a very basic SSR template based site and enrich with client side scripting for enhanced features and go from there. I would also add a service worker to cache all html which are fairly static and assets to reudce amount of hits to the server for these.

Totally agree with this approach. It is so easy to quickly end up with a bloated front-end when working with SPAs. Also, it is much more easier to go from simple SSR templates to something more complex than viceversa.

[superhawk610]

SEO is not only about meta tags. Content is the most important thing for SEO.

Huh, TIL. If this is the case then you're right, client-side markdown rendering probably isn't an option.

I do think there's a stronger argument to be made for a more barebones server-rendered approach, but I will say that I don't think going from templated SSR -> SPA is any easier than the alternative. I think it's a difficult transition both ways since any client-side stuff you've written in a non-React/Vue/Angular environment isn't really transferable to an environment with a framework and vice versa.

I've seen multiple people mention that adding a frontend framework like React will increase complexity, but I would argue it doesn't raise the bar for entry, it just requires a different skillset. Client-side UI interactions without a frontend framework like React introduce their own set of problems that are inherently solved when using a UI framework.

I guess I'm moving my vote from a dedicated SPA to a server-rendered SPA, ala Next or Gatsby. Best of both worlds 😄

[alexgarces]

@superhawk610

I think it's a difficult transition both ways since any client-side stuff you've written in a non-React/Vue/Angular environment isn't really transferable to an environment with a framework and vice versa.

Yes, you are right! It wouldn't be an easy task at all either. I was imagining this transition in a relatively early state (and in case of needing it).

I guess I'm moving my vote from a dedicated SPA to a server-rendered SPA, ala Next or Gatsby. Best of both worlds

Looks to me as a very feasible option as well! :)

[lacaulac]

@superhawk610 I don't know a lot about Server rendered SPAs but that seems to be a good solution.

[zacanger]

If the goal is to avoid big name company products, Preact is also an option, and it works with Next. The lead maintainer is a Google employee, but I believe the project is independent. I like Vue, but React or React-alikes would likely be more familiar to newcomers.

I've been keeping an eye on Svelte and it seems really neat, but I would think that for most contributors coming in, that would be a blocker, at least unless/until it's up there with with the big players (React, Ember, Vue, Angular).

[avickers]

I like Vue, but React or React-alikes would likely be more familiar to newcomers.

Let's go look at the number of Github stars on all of the projects under consideration. You're going to seriously argue that Vue isn't accessible enough? Arguing that Vue is somehow harder to pickup than React is an equally dubious assertion. There's probably a reason Vue ended up with more Github stars than React, in less time, and with millions of dollars less in marketing. That reason isn't that it provides a worse developer experience.

[varjmes]

"would be more likely familiar to newcomers" does not say anywhere in that sentence that Vue isn't accessible. Nor does it say Vue is harder.

It's evident you love Vue, and that's excellent! It's not as well known with beginners which is what I believe the above sentence was saying. Let's keep it chill pals 🌱

[toddself]

@Vickers if you could please tone down the aggressive attitude that would be fantastic. This is a place for friendly civilized chat, even if its about passionate topics like web frameworks. :)

[moogacs]

I am with Vue, what i really like in Vue and don't see in React is the code architecture

What about making a poll ?

[zacanger]

@avickers please don't misunderstand, I'm not saying I dislike Vue or think it's hard to learn. Vue is great, I really like Vue, and in some ways I think it's actually easier to pick up than React (especially the less fragmented ecosystem). It's just less widespread than React, at least for now (based on package download counts, Wappalyzer/BuiltWith stats, and GitHub's "Used By" count, not stars). That means that a new contributor to this project would probably be more likely to already be familiar with React than with Vue.

[avickers]

@zacanger I apologize for the snark.

Stack Overflow Developer survery 2019

I'll grant you that React is most loved by people that use it, and most desired by FEDs to add to their skillset, but Vue is second in both categories. After all of the different approaches that had been entertained here and given a fair shake, I guess I just felt Vue deserved better than a hand wave.

[venikx]

Both "Used By" and "Stars" are a bad metric to check the adoption rate and shouldn't really be used for the sake of arguments (imho). Both metrics have flaws. Stars displays "people like it" and used by only display adoption rate on Github (I believe?). Vue is big in Asia and I'm not sure how popular Github use is there.

Even if we decide to go with the Vue route, the fact it's supposed to be easier to pick up for beginners is an argument on it's own. React, Ember, Angular, whatever devs are able to pick it up fast and "new" frontend developers might have an easier time contributing (since it's supposed to be more approachable).

Can you do Github polls?

[cmmartti]

Can you do Github polls?

Any comment can be a poll. Just use the up/down vote emojis.

[zacanger]

There's also this https://app.gh-polls.com/ for more complex polls

[venikx]

Thanks, there ya go!

[workingjubilee]

I disagree foundationally with this poll.

1. FPtP is intolerable because it seeks to choose a winner who can acquire the most first-choice votes, not necessarily the most tolerable compromise, which makes it ideal for irritating people (just ask Americans, hey?). Which of course this poll defaults to. Informational video on more regarding this: https://www.youtube.com/watch?v=s7tWHJfhiyo
2. Relatedly, this poll has been loaded with many similar choices, necessarily distributing votes awkwardly amongst them. But also see point 8.
3. Many of the questions that have led to discussing Gatsby vs Next vs Preact vs yes even Vue are merely economic, as they involve tradeoffs of HD size vs CPU, and while many things have pointed at a future involving 100 more copies of entropic being hosted, the people footing the first bill right now for https://entropic.dev/ are @chrisdickinson and @ceejbot afaict. The reality of their hosting plans will logically have as much of a vote, if not more than anything, right now, and those are mere calculations that will not bend to the will of democracy.
4. Further, the reality of the economics of hosting plans is something I believe the initial 2 founders of this project have thought about more than me, so I would prefer to hear their reasoning about this before anything else, even if they were not ALSO footing the bill for entropic.dev
5. Again, it's going to be a site they are hosting, so if the poll says one way and they say another (for whatever reason), this has been a sham poll. If we pressure them to accept regardless, it has been a double sham, for it has been a poll that has betrayed our intent to help them. And we can now no longer know what they would've said in the absence of the poll because we pushed on anyways!
6. In addition, there have been arguments regarding "vision" and library compatibility with that, and I would like to hear what they have to say about how much anything indeed has to do with this, because all I had when I came here was the 30 minute talk and noticing some interesting patterns about npm usage.
7. Related to that, it's worth considering that Node went from an open software project to being sponsored by Joyent to being sponsored by the Node Foundation (which is apparently, in 2019, merging to make the OpenJS foundation? huh).
8. Part of why I said I'm in favor of more complete rendering libraries generally is because the differences between React and Vue are really an ocean of breadth smaller than their similarities.
9. Maybe we should chill and settle down in light of Vacation / Out of GitHub Jun 8 - Jun 10 #160

[olingern]

I feel like this discussion is jumping ahead of ourselves. We need a stable API first and then figure out how sessions will work on the site. As of #135 there are no JWT's for authentication so we will need to authorize access on the server for now.

Injecting React, Vue etc. for better interaction in the future ( I think ) would be a more pragmatic approach and put less pressure on having an API that is unchanging. Search engine indexing is also of concern.

Perhaps we can chat about a server side templating language / CSS framework as a next step after #135 merges?

---

Addendum:

Just some considerations from the requirements that @chrisdickinson posted:

Markdown is pre-rendered on publish. It is stored at rest as HTML suitable for inlining. (Storage is cheaper than CPU cycles.)

Injecting this and rendering after page load doesn't sound ideal. Sending this along from the server after retrieved from the database sounds more performant and better for search engine crawlers.

We want i18n support. My experience with rich client apps has been that you end up having to generate multiple different asset bundles for each target locale (or else do some kind of code-splitting.)

The website should look to what comes of #115.

[zacanger]

@workingjubilee

2: I think the options in the poll were taken from the most common suggestions in this issue thread.

3, 4, 5: Definitely! I don't think anyone thinks this poll is in any way binding, I think we're just trying to get a feel for what people are leaning towards, since the issue comments have gone on for a long while and it's hard to sum up everything that's happened.

9: I don't think a core maintainer being offline for a bit means folks need to stop contributing ideas! No one's trying to force a decision through, I think everyone here is just trying to figure out what might be best for this project.

@olingern

I somewhat agree, but right now there's a small but slowly growing bit of server-side rendered content already in the form of template strings, and I think one driving thing behind the discussion here is wanting to head that off and settle on a real, planned, and maintainable solution before we end up with lots of disorganized rendering hanging around.

Also regarding pre-rendering Markdown on package publish, I think there should be more discussion around whether that's ideal. That's trading off disk or s3 space and publish-time cycles for render-time cycles, and some of the debate in this thread has been about whether that's even the right thing to do.

[olingern]

I somewhat agree, but right now there's a small but slowly growing bit of server-side rendered content already in the form of template strings

@zacanger Totally agree. The template strings aren't ideal and I wanted to suggest using a template engine. A few suggestions:

• https://github.com/mozilla/nunjucks
  • Mozilla project, still being maintained
• https://github.com/olado/doT
  • No deps, but last commit 2017
• https://github.com/pugjs/pug
  • Defacto templating language, good docs, battle tested

---

RE S3 storage:

That's a hard problem. It would be neat if we could push to S3 and serve as a static site and then iframe the package HTML via the packages s3 static site URL. This would push the loading off to the user's browser and S3 at the expense of an additional network request. I'll admit maybe not practical.

Perhaps it's worth opening a thread on discourse to chat pros / cons of storing HTML vs doing it at runtime?

[zacanger]

@olingern I think starting a thread over there would be good!

[avickers]

Amazon S3 + Amazon CloudFront

[olingern]

Done. Discourse discussion here.

[venikx]

@workingjubilee
1: True, but that’s not the intention of the vote, it’s by no way binding. It’s just to get a feel of what people here are thinking, because we all have so many opinions it’s hard to see a summary.
2: I just put in the options I read on this thread. You can vote on multiple options either way.
3: See 1
4: See 1
5: Again, see 1
6: I don’t see what this has to do with the poll?
7: See 6
8: See 6
9: Disagree, I don’t think we should stop discussions, because one of the core members is on holiday.

If it wasn’t clear enough: My intention of the poll was not to set something in stone, but to get an idea of what people think of doing right with the information we got about the requirements. It’s by no means binding, especially since there is not clear “winner”.

@olingern
If it comes down templating language, I would fine with a mature one, which is actively maintained. No preference there, since I don’t have experience with templating language, but that’s fine.

I disagree with the point the discussion is getting ahead of itself, since that was the part of why the issue was opened: templating language VS something like Nextjs.

[venikx]

Seeing the discussions about how dirt cheap storage is, I’m thinking maybe gatsby would be a super viable approach. I kinda brushed of gatsby, because I was thinking it would cost a lot due to amount routes we would need for all the packages.

But if storage scales better than cpu cycles, Gatsby becomes actually a super viable option + if anyone wants to run their own it’s straightfoward since it’s “just html”. Accessible, straightforward and a friendly community ready to help as seen in: #107 (comment)

I’m unsure how two things work with gatsby though:

• Authorizarion
• Publish to registry would require redeploy of the FE if I’m not mistaken

[avickers]

Gatsby doesn't yet support incremental builds. Sounds like they think they are getting close.

Obviously rebuilding 800k pages with every publish would be the most expensive approach in terms of CPU. Maybe there is some clever way to break it up, like every namespace getting its own Gatbsy instance, that would mitigate this problem?

[superhawk610]

every namespace getting its own Gatbsy instance

That actually would work, the only difficulty there would be dynamically creating instances on namespace creation. Tbh I think incremental builds would essentially do this.

I still really want Gatsby to work since it will only require a rebuild on package creation/update, while Next will rerender each page every time it’s requested, but until they support incremental I don’t think it’s really viable.

I wonder if there’s some happy medium we could hit using Next - have an Express server running to serve the app, but whenever a route is first requested, render it and cache the HTML to disk and then just serve that on subsequent requests? Then any time a package is updated, we just delete the cached copy as part of the update process and it will be regenerated on the next request. For efficiency, we could even perform the render/cache on package deploy/update, effectively having our own Gatsby-esque solution. Thoughts?

To address @venikx other question, Gatsby apps are just basic React SPAs once they’re rehydrated so you can use any existing auth solution however you normally would (request a token from the backend and then store it locally and use it to authenticate outgoing requests).

[jlengstorf]

every namespace getting its own Gatsby instance

There's support for prefixing, so a URL structure of /:namespace/* would allow for an instance-per-namespace setup. Gatsby themes would allow for all common code to be rolled up into a single shared package to avoid the need to duplicate code between instances.

Incremental builds in Gatsby aren't too far off, either, so in the near- to mid-term this issue will resolve itself upstream.

[michaelpumo]

For your tech stack, I would certainly argue in favour for Vue JS with its SSR partner Nuxt JS.

It’s inspired by React’s use in Next, yet much simpler to get running with, easy HTML templating and CSS setup. I believe it to be a much more accessible setup that would welcome new contributors warmly.

In the spirit of this project, Vue (unlike React or Angular) is NOT back by a large company (Facebook and Google respectively). It was started by one person but is now a truly open source, community-led framework.

Lastly, if you wish to go with a CSS class based utility library I would recommend Tailwind over Tachyons.

That’s just my thoughts.

[paxelpixel]

phew, that's a long thread of framework debate.

just start with nunjuck, and vanilla js, css.

and add react later.

[evomedia247]

Keep it simple ... don't reinvent the wheel, and don't try and reach perfection on your first build.
Who really cares if its in Angular, React, Vue, or even Aurelia or Knockout or Jquery. I'd just say concentrate on being able to quickly refine based on your user behaviours is key at the start. Lets be honest an SPA feels slicker, but when your starting, just doing the simplest route to get something up is far better than creating some ultra complex Angular monster say.

Personally I'd build a simple prototype, throw up a super simple knockout.js based view model as it involves no build steps, I'd add in something like wasabi on a docker image to run AB tests and set up a load of goals, like completes a sign up form, links to the git hub repo. And start testing the users.

I'd add redirect functionality to attach tracking codes off URLs to segment the users and see where they are coming from to start measuring what is driving traffic, ie ads, talks, articles, social media etc.

If your doing a simple content only sire, then there are some decent headless cms systems like Crafter cms, or craft cms that take a few minutes to get up and running and then you get no major config or environment headaches.

Then build out from there, refine, refactor and renew

[Vickers]

Please remove me from this group.

…

On Thu, Jul 4, 2019, 10:15 AM evomedia247 ***@***.***> wrote: Keep it simple ... don't reinvent the wheel, and don't try and reach perfection on your first build. Who really cares if its in Angular, React, Vue, or even Aurelia or Knockout or Jquery. I'd just say concentrate on being able to react and refine based on your user behaviours. Lets be honest an SPA feels slicker, but when your starting, just doing the simplest route to get something up is far better than creating some ultra complex SPA. Personally I'd build a simple prototype, I often just throw up a super simple knockout.js based view model as it involves no build steps, I'd add in something like wasabi on a docker image to run AB tests and set up a load of goals, like completes a sign up form, links to the git hub repo. And start testing the users. Add redirects to add tracking codes to segment the users and see where they are coming from to start measuring what is driving traffic, ie ads, talks, articles, social media etc. If your doing a simple content only sire, then there are some decent headless cms systems like Crafter cms, or craft cms that take a few minutes to get up and running and then you get no major config or environment headaches. Then build out from there, refine, refactor and renew — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#107?email_source=notifications&email_token=AAHEX4EPSZ5FAMQ4NQPKPUDP5YAXZA5CNFSM4HSBNOZ2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZHQ53Y#issuecomment-508497647>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAHEX4DNV4IHKOWIDSTT423P5YAXZANCNFSM4HSBNOZQ> .

[toddself]

@Vickers we cannot remove you from this list -- you'll need to either use the mute thread link at the bottom of the email, or view the thread on github and click the unsubscribe button.

Sorry :(