Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Drizzle integration. #1027

Open
RajaARK99 opened this issue Sep 13, 2023 · 31 comments
Open

Drizzle integration. #1027

RajaARK99 opened this issue Sep 13, 2023 · 31 comments

Comments

@RajaARK99
Copy link

We need support drizzle integration.

@hayes
Copy link
Owner

hayes commented Sep 13, 2023

There is some initial experimentation around this here: #1028 but you can always just use drizzle manually, you don't need a plugin to use drizzle

@Enalmada
Copy link

@hayes thanks so much for the time you put into pothos. I just moved to Pothos and Drizzle and was really happy how easy builder.objectRef made things. Awesome there might be even tighter integration one day but it certainly is very nice even as it is.

// @/server/db/schema
export const UserTable = pgTable('user', {
    id: serial('id').primaryKey(),
   ....
});

export type User = InferSelectModel<typeof UserTable>;
export type UserInput = InferInsertModel<typeof UserTable>;
// user.model.ts
import { type User } from '@/server/db/schema';
import { builder } from '@/server/graphql/builder';

export const UserType = builder.objectRef<User>('User');

UserType.implement({
  fields: (t) => ({
    id: t.exposeID('id'),
    ...
  }),
});

builder.queryField('me', (t) =>
  t.field({
    type: UserType,
    nullable: true,
    resolve: (root, args, ctx) => {
      return new UserService().me(ctx);
    },
  })
);

@hayes
Copy link
Owner

hayes commented Sep 23, 2023

Nice, glad it's working well for you! If you've got any advanced cases where you are doing joins/dataloading/nested selects or anything like that, id be curious to see how you're doing it or what the pain points there are.

The current plan with the drizzle plugin is to lean on a combination of drizzles relational query builder, and data-loaders similar to what Prisma does. This kinda gets away from being close to SQL which is one of the things that makes drizzle interesting, but it's a lot easier to build out that way.

@Enalmada
Copy link

@hayes I am currently only using it in simple cases but happy to share what I am doing. Here are some helpers I am currently prototyping which make it more prisma like and make the service code a bit cleaner when dealing with things like pagination: https://github.com/Enalmada/drizzle-helpers

Here is a demo project where this can be seen in action: https://github.com/Enalmada/nextjs-boilerplate

@liquiad
Copy link

liquiad commented Sep 26, 2023

I would love to see the Drizzle plugin make it out. Prisma leaves a lot to be desired regarding performance. N+1 queries with Prisma will add exponentially more round trips to your database, which is a problem Drizzle simply doesn't have. Properly transforming a GraphQL query's relations to left joins in a single SQL statement is the dream.

As of 10/14/23, Prisma is part of the way there now lol https://github.com/prisma/prisma/releases/tag/5.4.0

@arolson101
Copy link

FWIW, I got it working using drizzle-graphql and AddGraphQLPlugin

import { buildSchema } from 'drizzle-graphql';
import AddGraphQLPlugin from '@pothos/plugin-add-graphql';

const conn = postgres(env.DATABASE_URL);
const db = drizzle(conn, { schema });
const { entities } = buildSchema(db);

const builder = new SchemaBuilder({
  plugins: [AddGraphQLPlugin],
});

const User = builder.addGraphQLObject(entities.types.UsersItem, {
  name: 'User',
  description: 'description for User',
  fields: (t) => ({
    id: t.exposeID('id'),
  }),
});

const UserUpdate = builder.addGraphQLObject(entities.types.UsersSelectItem, {
  name: 'UserUpdate',
  fields: (t) => ({
    id: null,
  }),
});

builder.queryType({
  fields: (t) => ({
    users: t.field({
      type: [User],
      resolve: async (_root, _args, { db }) => {
        const users = await db.query.users.findMany();
        return users;
      },
    }),
  }),
});

export const schema = builder.toSchema();

@hayes
Copy link
Owner

hayes commented Jul 31, 2024

I know this issue has been open for a long time, but I finally got the initial version of the plugin published!

https://pothos-graphql.dev/docs/plugins/drizzle

It's not production ready, but I'd love to get some people to try it out, and give me some feedback.

If you are using Pothos with drizzle already, or have any interest in playing with this, let me know! I am more than happy to help get things set up for some testing!

@dantrain
Copy link

dantrain commented Aug 4, 2024

@hayes this is so awesome, thanks for all your efforts!

Some quick feedback for you: I went to try the plugin out on a project that uses Cloudflare D1. For this I have to construct the Drizzle client in the request context in order to have access to the "Binding". I then pass it into the context for use in resolvers.

With the Drizzle plugin it seems I must pass the Drizzle client when constructing the Builder. Could there be a way to tell the plugin to get it from the context instead?

@hayes
Copy link
Owner

hayes commented Aug 4, 2024

Added an option so you can do something like this:

export const builder = new SchemaBuilder<PothosTypes>({
  plugins: [RelayPlugin, DrizzlePlugin],
  drizzle: {
    client: (_ctx) => drizzle(client, { schema }),
    schema,
  },
});

@dantrain
Copy link

Finally had a chance to give the plugin a go! The option to use the Drizzle client from context worked great. Some observations for you:

  • Passing a single argument to t.relatedConnection gave me a type error: reviews: t.relatedConnection("reviews"),
    I got it to work by passing an empty query option: reviews: t.relatedConnection("reviews", { query: {} }),

  • Similarly, passing no arguments to query() in a t.drizzleConnection resolver didn't work for me, I had to pass an empty object:

builder.queryField("movies", (t) =>
  t.drizzleConnection({
    type: "movies",
    resolve: async (query, _parent, _args, { db }) =>
      db.query.movies.findMany(query({})),
  }),
);
  • I had a mutation with an id argument and I was using decodeGlobalId from @pothos/plugin-relay to get back the primary key, but with the Drizzle plugin the decoded ID was prefixed with S: which I had to remove.

  • Finally, I haven't thought this through too much but previously I was able to co-locate the Drizzle table definition and the Pothos object in the same file. This worked because I was only importing the types when instantiating the builder and passing them as Objects, and this is stripped out for runtime. But now I'm importing the Drizzle schema, the builder is not instantiated in time. Totally makes sense, but a shame, I thought the co-location was neat.

Hope this helps!

@hayes
Copy link
Owner

hayes commented Aug 13, 2024

Thanks for the feedback @dantrain

I don't have a good suggestion to fix to co-location thing. There are hacky workarounds, but they probably aren't worth it.

  • I've fixed the issues related to query and relatedConnection having required arguments
  • I've update the node ID encoding to not include the type tags
    • I also added a parse methods so that t.arg.globalId({ for: NodeRef }) will parse the ID, but it will be parsed as an object, and you end up with something like arg.id.id.id if you your arg and column are both "id", or more generally: arg.argName.id.columnName. This is required to make multi-column ids work smoothly.
    • using decodeGlobalId you should get just the column value as a string

I'm curious if this is generally going in the right direction, and if its an improvement over using drizzle manually.

@dantrain
Copy link

Brilliant, the fixes work great thank you!

I think the relations and connections are definitely a nice improvement for me over using Drizzle manually.

A few years back before I discovered Pothos I used Nexus and nexus-plugin-prisma for a big project. It allowed us to move quickly but it felt very magic and resulted in some inefficient queries.

I tried out Drizzle to have more control over the queries than Prisma and retain the type safety, and really liked it. Defining the database schema in code is great, and unlike Prisma it works on an edge runtime like Cloudflare Workers etc.

I was missing some of the convenience though. I think it's getting to a good middle ground with Drizzle's relational API and this new plugin.

Once the Drizzle relational API v2 comes out it would be useful to have some helpers to create input types, similar to your "Prisma Utils". Excited for the future!

@hayes
Copy link
Owner

hayes commented Aug 13, 2024

Once the Drizzle relational API v2 comes out it would be useful to have some helpers to create input types, similar to your "Prisma Utils". Excited for the future!

It's not documented and I am not sure I want to keep it, but there you this already exists (sortof).

Screenshot 2024-08-13 at 11 42 28 AM

Using https://orm.drizzle.team/docs/graphql

you can do something like const UsersFilters = builder.drizzleGraphQLFilters('users', entities.inputs.UsersFilters). You just need to setup drizzle-graphql and the add-graphql plugin.

My experience was that this made type-checking very slow, and it was WAY too easy to create dangerous DB queries, but its there if you want to try it (but I might remove it before creating a stable release)

@RajaARK99
Copy link
Author

@hayes Typescript will not work correctly when using drizzleFieldWithInput.

I am installing both @pothos/plugin-with-input and drizzle plugin.

builterconfig

import SchemaBuilder from "@pothos/core";
import DrizzlePlugin from "@pothos/plugin-drizzle";
import { buildSchema } from "drizzle-graphql";
import RelayPlugin from "@pothos/plugin-relay";
import { DateResolver, DateTimeISOResolver } from "graphql-scalars";
import WithInputPlugin from "@pothos/plugin-with-input";
import ZodPlugin from "@pothos/plugin-zod";
import ErrorsPlugin from "@pothos/plugin-errors";
import { db } from "./db/db";
import * as schema from "./db/schema";

const { entities } = buildSchema(db);

export interface PothosTypes {
  DrizzleSchema: typeof schema;
  Scalars: {
    Date: {
      Input: Date;
      Output: Date;
    };
    DateTime: {
      Input: Date;
      Output: Date;
    };
  };
}

const builder = new SchemaBuilder<PothosTypes>({
  plugins: [
    RelayPlugin,
    // ZodPlugin,
    WithInputPlugin,
    ErrorsPlugin,
    DrizzlePlugin,
  ],
  drizzle: {
    client: db,
    schema,
  },
  relay: {},
  withInput: {
    typeOptions: {},
    argOptions: {},
  },
  // zod: {
  //   validationError: (zodError, args, context, info) => {
  //     return zodError;
  //   },
  // },
  errors: {
    defaultTypes: [],
  },
});
builder.objectType(Error, {
  name: "Error",
  fields: (t) => ({
    message: t.exposeString("message"),
  }),
});
builder.addScalarType("Date", DateResolver);
builder.addScalarType("DateTime", DateTimeISOResolver);

builder.queryType();
builder.mutationType({
  fields: (t) => ({
    createUser: t.fieldWithInput({
      input: {
        email: t.input.string({ required: true }),
        name: t.input.string({ required: true }),
      },
      type: "String",
      resolve: async (parent, args, ctx) => {
        return "Hello World";
      },
    }),
  }),
});
export { builder, entities };

Can you document this with an example.

Screenshot 2024-09-09 161506

Screenshot 2024-09-09 161512

import { db } from "../db/db";
import { builder, entities } from "../builder";
import { users } from "../db/schema";
import { z } from "zod";
import { emailZodSchema, stringZodSchema } from "../lib/zod-schema";

const usersRef = builder.drizzleObject("users", {
  name: "User",
  description: "User field",
  fields: (t) => ({
    id: t.exposeID("id"),
    email: t.exposeString("email"),
    name: t.exposeString("name"),
    createdAt: t.expose("createdAt", {
      type: "DateTime",
    }),
    updatedAt: t.expose("updatedAt", { type: "DateTime" }),
  }),
});

builder.mutationFields((t) => ({
  createUser: t.drizzleFieldWithInput({
    type: usersRef,
    input: {
      email: t.input.string({ required: true }),
      name: t.input.string({ required: true }),
    },
    resolve: (parent, { input }, ctx) =>
      db.insert(users).values(args).returning(),
  }),
}));

@hayes
Copy link
Owner

hayes commented Sep 9, 2024

@RajaARK99 you are missing the query arg, it's query, parent, arga, ctx, info for drizzle fields

@RajaARK99
Copy link
Author

RajaARK99 commented Sep 9, 2024

@hayes I also update it before but it not work

@hayes
Copy link
Owner

hayes commented Sep 9, 2024

thanks for reporting this! I've identified and fixed the issue, I'll have the release out in a minute

@hayes
Copy link
Owner

hayes commented Sep 9, 2024

@RajaARK99 fix should be available in 0.4.3

@alexanderniebuhr
Copy link

Finally, I haven't thought this through too much but previously I was able to co-locate the Drizzle table definition and the Pothos object in the same file. This worked because I was only importing the types when instantiating the builder and passing them as Objects, and this is stripped out for runtime. But now I'm importing the Drizzle schema, the builder is not instantiated in time. Totally makes sense, but a shame, I thought the co-location was neat.

What is the issue with co-location here, I'm just planning to adopt Pothos, and was trying to come up with a concept, where Drizzle schema and Pothos builder code is co-located, would that still work? And if not why, and what are the alternatives?

@Hebilicious
Copy link

Congratulations on releasing this, awesome work as usual !

Unfortunately I'm no longer working on the project I was working before that used drizzle and Pothos together, so I won't be able to test this in the near term. But imo Drizzle + Pothos is the best choice to work with graphql, sql and typescript, so I will eventually take it for a spin one day.

I went through the doc and would suggest to adding a section that explains how to use Drizzle without the plugin (similar to the Prisma section).
Using Drizzle without the plugin worked great for me and the only paint point was to have to write some complex helpers functions that are working properly with Typescript, such as parsing the info object to get the exact list of selected fields and relations to build the drizzle query.

I think illustrating how much the plugin does compare to using it without is great both for understanding Pothos more and the value the plugin provides.

@rawkode
Copy link

rawkode commented Nov 13, 2024

I've been experimenting with this and I've run into a problem.

❯ deno run -A read-model/main.ts
Listening on tcp://0.0.0.0:8000
ERR Error: Field subtitle not resolved in initial query and no primary key found for type episode
    at file:///var/home/rawkode/.cache/deno/npm/registry.npmjs.org/@pothos/plugin-drizzle/0.5.1/esm/index.js:101:23
    at executeField (file:///var/home/rawkode/.cache/deno/npm/registry.npmjs.org/@graphql-tools/executor/1.3.1/esm/execution/execute.js:317:24)
    at executeFields (file:///var/home/rawkode/.cache/deno/npm/registry.npmjs.org/@graphql-tools/executor/1.3.1/esm/execution/execute.js:265:28)
    at collectAndExecuteSubfields (file:///var/home/rawkode/.cache/deno/npm/registry.npmjs.org/@graphql-tools/executor/1.3.1/esm/execution/execute.js:694:23)
    at completeObjectValue (file:///var/home/rawkode/.cache/deno/npm/registry.npmjs.org/@graphql-tools/executor/1.3.1/esm/execution/execute.js:684:12)
    at completeValue (file:///var/home/rawkode/.cache/deno/npm/registry.npmjs.org/@graphql-tools/executor/1.3.1/esm/execution/execute.js:452:16)
    at file:///var/home/rawkode/.cache/deno/npm/registry.npmjs.org/@graphql-tools/executor/1.3.1/esm/execution/execute.js:320:49
    at eventLoopTick (ext:core/01_core.js:175:7)
    at async file:///var/home/rawkode/.cache/deno/npm/registry.npmjs.org/@graphql-tools/executor/1.3.1/esm/execution/promiseForObject.js:15:35
    at async Promise.all (index 0) {
  message: "Field subtitle not resolved in initial query and no primary key found for type episode",
  path: [ "episode", "subtitle" ],
  locations: [ { line: 3, column: 5 } ],
  extensions: [Object: null prototype] {}
}

My resolver looks like this:

builder.queryType({
	fields: (t) => ({
		episode: t.field({
			type: episodeRef,
			args: {
				code: t.arg({
					type: 'String',
					required: true,
				}),
				showId: t.arg({
					type: 'String',
					required: true,
				}),
			},
			resolve: (_root, args, _ctx) =>
				db.query.episodesTable.findFirst({
					where: and(
						eq(dataSchema.episodesTable.showId, args.showId),
						eq(dataSchema.episodesTable.code, args.code),
					),
				}).execute(),
		}),

and I've confirmed that the query is returning:

{
  showId: "SG1",
  code: "S04E06",
  title: "Window of Opportunity",
  subtitle: "A classic episode",
  description: "The team is stuck in a time loop."
}

I'm not sure where I've went wrong :(

@hayes
Copy link
Owner

hayes commented Nov 13, 2024

complete side note, but that is one of my favorite episodes of any tv show

@hayes
Copy link
Owner

hayes commented Nov 13, 2024

can you share the the relevant episode implementation and the query you are running?

@rawkode
Copy link

rawkode commented Nov 13, 2024

@hayes
Copy link
Owner

hayes commented Nov 13, 2024

You should be using t.drizzleField instead, although I am a little surprised it doesn't work without it, so there is likely something a slightly off in the plugin implementation, but try switching your query field to use t.drizzleField instead, and see if that works

@rawkode
Copy link

rawkode commented Nov 13, 2024

Yeah, I had tried that too; unfortunately.

❯ deno run --allow-all read-model/main.ts 
Listening on tcp://0.0.0.0:8000
ERR Error: Field title not resolved in initial query and no primary key found for type episode
    at file:///var/home/rawkode/.cache/deno/npm/registry.npmjs.org/@pothos/plugin-drizzle/0.5.1/esm/index.js:101:23
    at executeField (file:///var/home/rawkode/.cache/deno/npm/registry.npmjs.org/@graphql-tools/executor/1.3.1/esm/execution/execute.js:317:24)
    at executeFields (file:///var/home/rawkode/.cache/deno/npm/registry.npmjs.org/@graphql-tools/executor/1.3.1/esm/execution/execute.js:265:28)
    at collectAndExecuteSubfields (file:///var/home/rawkode/.cache/deno/npm/registry.npmjs.org/@graphql-tools/executor/1.3.1/esm/execution/execute.js:694:23)
    at completeObjectValue (file:///var/home/rawkode/.cache/deno/npm/registry.npmjs.org/@graphql-tools/executor/1.3.1/esm/execution/execute.js:684:12)
    at completeValue (file:///var/home/rawkode/.cache/deno/npm/registry.npmjs.org/@graphql-tools/executor/1.3.1/esm/execution/execute.js:452:16)
    at completeValue (file:///var/home/rawkode/.cache/deno/npm/registry.npmjs.org/@graphql-tools/executor/1.3.1/esm/execution/execute.js:426:27)
    at completeListItemValue (file:///var/home/rawkode/.cache/deno/npm/registry.npmjs.org/@graphql-tools/executor/1.3.1/esm/execution/execute.js:572:29)
    at completeListValue (file:///var/home/rawkode/.cache/deno/npm/registry.npmjs.org/@graphql-tools/executor/1.3.1/esm/execution/execute.js:553:13)
    at completeValue (file:///var/home/rawkode/.cache/deno/npm/registry.npmjs.org/@graphql-tools/executor/1.3.1/esm/execution/execute.js:438:16) {
  message: "Field title not resolved in initial query and no primary key found for type episode",
  path: [ "showEpisodes", 0, "title" ],
  locations: [ { line: 3, column: 5 } ],
  extensions: [Object: null prototype] {}
}
^C⏎                                                                                                                                                                                                
projects/rawkode.academy/episodes-service on  feat/episodes [!?] via 🦕 v2.0.2 on ☁️  [email protected] took 20s 
❯ git diff
diff --git a/projects/rawkode.academy/episodes-service/read-model/main.ts b/projects/rawkode.academy/episodes-service/read-model/main.ts
index 40968bd..e3d9bf8 100644
--- a/projects/rawkode.academy/episodes-service/read-model/main.ts
+++ b/projects/rawkode.academy/episodes-service/read-model/main.ts
@@ -53,7 +53,7 @@ builder.asEntity(episodeRef, {
 
 builder.queryType({
        fields: (t) => ({
-               episode: t.field({
+               episode: t.drizzleField({
                        type: episodeRef,
                        args: {
                                code: t.arg({
@@ -65,7 +65,7 @@ builder.queryType({
                                        required: true,
                                }),
                        },
-                       resolve: (_root, args, _ctx) =>
+                       resolve: (_query, _root, args, _ctx) =>
                                db.query.episodesTable.findFirst({
                                        where: and(
                                                eq(dataSchema.episodesTable.showId, args.showId),

@hayes
Copy link
Owner

hayes commented Nov 13, 2024

I can probably try to debug more later tonight, but will be in meetings most of the day today.

You are probably running into this issue: drizzle-team/drizzle-orm#2932 which would prevent attempting to reload the record (which is why you are seeing no primary key found).

But the real issue is something else, because it shouldn't need to reload the record in the first place.

Are you able to share the query being executed when you see this error?

@rawkode
Copy link

rawkode commented Nov 13, 2024

query {
  showEpisodes(showId: "SG1") {
    title
  }
  
  episode(showId:"SG1", code:"S04E06") {
    title
  }
}

Both fail:

{
  "errors": [
    {
      "message": "Unexpected error.",
      "locations": [
        {
          "line": 3,
          "column": 5
        }
      ],
      "path": [
        "showEpisodes",
        0,
        "title"
      ]
    },
    {
      "message": "Unexpected error.",
      "locations": [
        {
          "line": 6,
          "column": 5
        }
      ],
      "path": [
        "episode",
        "title"
      ]
    }
  ],
  "data": {
    "showEpisodes": [
      {
        "title": null
      }
    ],
    "episode": {
      "title": null
    }
  }
}

@hayes
Copy link
Owner

hayes commented Nov 14, 2024

@rawkode cloned your project and tested this out.

if you use query which you ALWAYS should/need to, everything works as expected:

builder.queryType({
	fields: (t) => ({
		episode: t.drizzleField({
			type: episodeRef,
			args: {
				code: t.arg({
					type: 'String',
					required: true,
				}),
				showId: t.arg({
					type: 'String',
					required: true,
				}),
			},
			resolve: (query, _root, args, _ctx) =>
				db.query.episodesTable.findFirst(query({
					where: and(
						eq(dataSchema.episodesTable.showId, args.showId),
						eq(dataSchema.episodesTable.code, args.code),
					),
				})),
		}),
		showEpisodes: t.drizzleField({
			type: [episodeRef],
			args: {
				showId: t.arg({
					type: 'String',
					required: true,
				}),
			},
			resolve: (query, _root, args, _ctx) =>
				db.query.episodesTable.findMany(query({
					where: eq(dataSchema.episodesTable.showId, args.showId),
				})),
		}),
	}),
});

@rawkode
Copy link

rawkode commented Nov 14, 2024

Thank you so much, @hayes ! Appreciate the assist 👍🏻

@hayes
Copy link
Owner

hayes commented Nov 14, 2024

The core issue here is Pothos doesn't currently trust you to not select the wrong thing.

If you don't use the query helper, it assumes you could be doing something like:

db.query.episodesTable.findMany({
	where: eq(dataSchema.episodesTable.showId, args.showId),
	select: {
			extras: {
					title: sql('password')
			}
	}
})

Using query doesn't really prevent this entirely, but pothos can track what was actually selected, and map the results to that selection.

This is mostly used to ensure that if you have something like:

query {
  show(id: "SG1") {
    oldEpisodes: episodes(order: ASC) { title }
    newEpisodes: episodes(order: DESC) { title }
  }
}

Pothos can inject the query for one of those 2 fields, and know it was loaded, and deffer loading the other one to the nested resolver.

For simple columns that are "exposed" its probably safe to assume we don't need to track this mapping, but the expose helpers currently use the same mechanics as relations, and have the same restrictions

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

9 participants