File structure by roles, not domain on top level

[mizchi]

I have opinion and question about the blitz-generated file structure.

I'm primarily a front-end engineer using next.js and also have a modicum of rails experience.

My opinion.

In Rails, the resources generated by rails generate look like this

app/models/(resourceName)/...
app/views/(resourceName)/...

I think this makes sense because when I write code, I think about whether it's a model or a view first.

But there is a known problem with this style: it's hard to represent a model that isn't tied to a table. This causes people unfamiliar with the rails style to write their domain logic in util. Also, with the front-end idea, the problem of not being able to express affiliation comes from the large number of views where no model exists.

In the current boilerplate generated by blitz, I am concerned that the domain(resource) and the top level components are mixed in same directory, and that it will be difficult to know what the object we are trying to modify belongs to.

For example, imagine a blog post with a bunch of comments. It's up to the implementer to decide which domain's View the CommentForm in the article page will be implemented in. It's more complicated when you have a transaction with multiple resources.

In my opinion, the code to generate blitz generate all ... will be like this.

• app/resources/(resourceName)/{mutations, queries}
• app/pages/(resourceName)/{index.ts, [id]/...}
• app/components/forms/(resourceName)Form.tsx

reason

• I thought it would be easier to modify the logic if the resource definitions were hanging under one structure.
• I feel that splitting pages into domains is a poor prospect. It's also a leap of faith as next.js etiquette
• In order to replicate the Rails development experience, the form is tightly coupled to the domain so we could generate it. As for the View, I thought it would be better to keep it naive and flexible.

It will cause big flat domain. Apart from this, there may be a way to split the domain. See #1141

Question.

• I'd like to know if there's a valid reason why this is happening now.
• I would like to know if I can send a PR to add the generator / or if there is a way to customize it on the user side.

[svobik7]

Hi @mizchi , thank you for sharing your idea 👍

I personally like domain structure (blitz structure) more than mentioned feature like structure.

I found much much more simplier to manage team work, PR, commits, changes when everything what is related is grouped in one folder.

So for me the structure like following wins:

tasks
— components
— models
projects
— components
— ...

But I agree with your comment about:

In the current boilerplate generated by blitz, I am concerned that the domain(resource) and the top level components are mixed in same directory, and that it will be difficult to know what the object we are trying to modify belongs to.

Before I used Blitz I was used to following structure:

app
— components
— modules
—— tasks
——— components
——— pages
—— projects
——— components
——— pages
— pages

Maybe this could solve potential confusion of mixed app stuff and domain stuff.

[rama-kairi]

It's Great a Good Idea..

[svobik7]

I've just updated my PRs about app/modules context #1231.

During implementation I noticed that one thing could be confusing after this PR merge. It is about using generator with context option together. It will work like this:

1. blitz generate project --> app/modules/projects
2. blitz generate admin/project --> app/admin/projects
3. blitz generate project --context=admin --> app/admin/projects

I was thinking about only one way of specifying context to be able to create nested folders:

1. blitz generate project --> app/modules/projects
2. blitz generate admin/project --> app/modules/admin/projects
3. blitz generate project --context=admin --> app/admin/projects

What do you think guys?

[flybayer]

So I'm thinking instead of setting modules as the default context like in your PR, we should change the context root to be app/modules/ instead of app/ (and then leave default context blank). Then we could add a flag like --root that could change the root to be anything. Default value is app/modules/.

[svobik7]

@flybayer that is a great idea! I will update my PR with that approach 👍

[svobik7]

@flybayer main + docs PRs updated 🚀

[mizchi]

Perhaps we each have our own opinion by scale. How about define them by blitz.config.js?

For example, blitz.config.js has generatePath option.

// this file is js but I wrote types for explanation.
type GenerateContext = {
    name: string;
    kind: "pages" | "form" | "queries" | "mutations";
    fileName: string;
    // ...other contexts
}

module.exports = {
  // ...
  generatePath({name, kind, fileName}: GenerateContext) {
     if (kind === "pages") { // my favorite single routes definition
        return `app/pages/${name}/${fileName}}`;
     }
     return `app/${name}/${kind}/${fileName}`; // default
  }
}

(and blitz can reallocate them if we can)

But in this case, blitz install have to need more abstract implementation...

[flybayer]

This is a good idea!

This is very much related to our custom templates feature that's in progress. You are basically wanting to customize the actual generators beyond just the templates, which I also want to do.

@aem, you have any thoughts on this?

[flybayer]

Thanks for this discussion @mizchi!

The reason we have the current file structure is that it's significantly more scalable than how Rails does it. For a lot of smaller apps, it doesn't matter either way. But we want to set you up with a default architecture that will scale very well instead of something that you will have to totally change at some point.

All large apps that I'm aware of end up using the file structure we have now in Blitz.

Here's an article from Shopify on why they changed away from the default file structure in their Rails app.

The first issue they chose to address was code organization. At this time, our code was organized like a typical Rails application: by software concepts (models, views, controllers). The goal was to re-organize it by real-world concepts (like orders, shipping, inventory, and billing), in an attempt to make it easier to locate code, locate people who understand the code, and understand the individual pieces on their own. Each component would be structured as its own mini rails app, with the goal of eventually namespacing them as ruby modules. The hope was that this new organization would highlight areas that were unnecessarily coupled.

And here's an interesting follow-up Shopify article on Enforcing Modularity in Rails Apps with Packwerk.

[aem]

So reading through the answers here, and having participated in most of the architecture discussions we've had over the last ~8 months (wow!), I've got a couple of thoughts:

1. A big concern early on was balancing separation of concerns, colocating files that change together, and complexity of the folder structure. These things all pull at each other, so finding a balance there is tricky. Personally, I think /app/modules/tasks/pages/tasks/index.tsx is a really long file path, it's a lot of folders to navigate in my file browser, and the name of modules is especially confusing because module is already an overloaded term in the JS/TS world. As a bare minimum I think we should bikeshed about the name a bit more.
2. If we move forward with this, it seems like an obvious candidate for a blitz.config.js option. We can put the argument in there by default so it remains discoverable and have a comment about its implications. Hard-coding this into the generator framework removes a lot of the benefit of the flexibility of the generators. It'll also interfere with the path generation assumptions we have to make when using custom templates.

Here's my proposal, let me know how you all feel:

// blitz.config.js
{
  // ...
  // The default directory to nest feature folders within the `app/` directory.
  // By default, all feature directories are created immediately inside `app/`.
  generatorPath: '.',
  // ...
}

And then all generator classes that output files must incorporate the generatorPath argument. We could enforce this by exposing a getGeneratorPath method in the base generator, and then, in the write phase, check that getGeneratorPath has been called, and throwing an exception if some bypass flag was not set (similar to how we disable the conflict checker).

[flybayer]

Yeah, I'm totally open to another name, but I do think an extra folder for grouping these files is ideal.

One thing we can do to reduce nesting, is automatically nest urls based on the folder structure, instead of having pages/ always be root. So your example path would then be /app/modules/tasks/pages/index.tsx with url of /tasks. We'd also have a way to escape back to the root url when needed.

Adding the config option is great. I think we should put the whole path there though (app/ instead of .)

[svobik7]

As a bare minimum I think we should bikeshed about the name a bit more.

Another names came to my mind are: sections, units, features, segments

Personally sections is the best option I think.

One thing we can do to reduce nesting, is automatically nest urls based on the folder structure...

I was about to suggest the same approach about reducing nesting. I find tasks/pages/tasks/index.tsx a little bit confusing so tasks/pages/index.tsx is great option here. As nextjs offers rewrites I see no issue with this approach 👍

I think we should put the whole path there though (app/ instead of .)

Agree with config option containing whole path.

[svobik7]

I want to pin this idea that emerged during this discussion:

One thing we can do to reduce nesting, is automatically nest urls based on the folder structure, instead of having pages/ always be root.

So instead of /app/tasks/pages/tasks/index.tsx would be /app/tasks/pages/index.tsx with url of /tasks. I pretty like this idea and do not see any issue about it (as nextjs provides custom rewrites for transforming final url).

What do you think guys? @flybayer @aem

[flybayer]

After trying the app/modules/* option and thinking about this more, I really don't like that too much.

So another idea. The root problem is too many disparate files inside app/. Another way to solve this is move those other folders out of app up to the top level.

So instead of this

├── app
│   ├── api
│   │   ├── _health.ts
│   │   └── syncToClever.ts
│   ├── components
│   │   ├── Button.tsx
│   │   ├── ErrorComponent.tsx
│   │   ├── Form.tsx
│   │   ├── LabeledDateField.tsx
│   │   ├── LabeledSelectField.tsx
│   │   └── LabeledTextField.tsx
│   ├── customers
│   │   ├── mutations
│   │   │   ├── createCustomer.ts
│   │   │   ├── deleteCustomer.ts
│   │   │   └── updateCustomer.ts
│   │   ├── queries
│   │   │   ├── getCustomer.ts
│   │   │   └── getCustomers.ts
│   │   └── validations.ts
│   ├── hooks
│   │   └── useCurrentUser.ts
│   ├── layouts
│       └── AppLayout.tsx

Change to this:

├── app/
│   ├── users/ ...
│   ├── users/ ...
│   ├── customers/
│       ├── mutations
│       │   ├── createCustomer.ts
│       │   ├── deleteCustomer.ts
│       │   └── updateCustomer.ts
│       ├── queries
│       │   ├── getCustomer.ts
│       │   └── getCustomers.ts
│       └── validations.ts
├── api/
│   ├── _health.ts
│   └── syncToClever.ts
├── components/
│   ├── Button.tsx
│   ├── ErrorComponent.tsx
│   ├── Form.tsx
│   ├── LabeledDateField.tsx
│   ├── LabeledSelectField.tsx
│   └── LabeledTextField.tsx
├── hooks/
│   └── useCurrentUser.ts
├── layouts/
│   └── AppLayout.tsx

[svobik7]

I am not sure if this is better than current structure. I don't like keeping only "modules" inside app folder and moving other stuff like "api, components, ..." (which are still related to app implementation) on the same level as db, public, ... folders.

For me the initial problem with current structure is that inside app I see "modules" spread among common app stuff like components, hooks, etc. it makes it harder to read in my opinion. Therefore the first thing I did with my first blitz app was moving any "module stuff" under modules folder to keep it readable. I agree with @aem that this way of structure makes paths one level longer/deeper.

So I think about the initial problem once again and came up with an idea of opposite way = keeping "modules" spread inside app folder but move common stuff inside own directory:

├── app
│   ├── api
│   │   ├── _health.ts
│   │   └── syncToClever.ts
│   ├── common
|   │   ├── components
|   │   │   ├── Button.tsx
|   │   │   ├── ErrorComponent.tsx
|   │   │   ├── Form.tsx
|   │   │   ├── LabeledDateField.tsx
|   │   │   ├── LabeledSelectField.tsx
|   │   │   └── LabeledTextField.tsx
|   │   ├── hooks
|   │   │   └── useCurrentUser.ts
|   │   └── layouts
|   │       └── AppLayout.tsx
│   ├── customers
│   │   ├── mutations
│   │   │   ├── createCustomer.ts
│   │   │   ├── deleteCustomer.ts
│   │   │   └── updateCustomer.ts
│   │   ├── queries
│   │   │   ├── getCustomer.ts
│   │   │   └── getCustomers.ts
│   │   └── validations.ts

I am not sure if common folder name is the best name but it describes my thoughts the best.

[aem]

I think what we're finding with the above comments is:

1. There is no top-level folder structure that makes sense for everyone, each developer/app is different
2. The naming of this thing is hard and shouldn't be dictated by us.

This is driving me to pretty firmly believe that we should add the config option, set it to app/ by default, and then in the docs give a few options of how it could be used and what the implications are but be totally unopinionated about it otherwise

[flybayer]

Yeah this sounds like the best approach for now. We can still change the default later if we get enough consensus and/or better ideas.

[svobik7]

Agree. Let developers decide on their own. So we can close #1231 with no merge for now, right?

[flybayer]

@svobik7 correct

[rizrmd]

Having read through previous comments, made me question more:

• Can we make app folder structure recursive ?
  • Like having another app folder inside current app. So we can have modules, sections or whatever you may have in mind.
• Can we have more than one app folder in root ?
• Can we move blitz specific folder (app/pages, app/queries) ?
  • Some developer may have a thinking like: “Can we extract some common queries into root ?”
• Can we repackage an app folder so we can easily re-use it in another blitzjs project ?
  • How do we extend repackaged app folder ?
  • Is repackaged app allowed to depends on another repackaged app ?
  • How do we update it ?

Sorry if my questions sounds weird, english is not my native 🙏...

[rizrmd]

I think repackaged app concept is especially beautiful. We can have something like ActiveAdmin that we can just plug and remove whenever we like.

Or maybe install full blown CMS (Content Management System) app to enhance our current business app.

IMO this feature will need a monorepo system to be really flexible.

[flybayer]

• You can move blitz specific folders however you like inside app/
• You could repackage various folders in app/ by using git submodules and then share them across apps.

[rizrmd]

What if we have dependencies in one app that is not required in another app ? both dependencies must be installed in current version of blitz.

It’s ok when the difference is just one or two packages. But it will be problematic when we have dozen of different packages.

[flybayer]

Yeah, that's a hard problem to solve! Open to ideas :)

[rizrmd]

Well, it's one of the reason why people choose to use monorepo, we can have multiple package.json in different folder. Since this problem will be inevitable when blitz project grows, it will be better if we have opinion / best practice to guide people in.

I really want to work on this, but my current gig takes too much of my time.