Introduce new experimental DSL for Postgrest Columns

[jan-tennert]

What kind of change does this PR introduce?

Feature.

What is the new behavior?

Introduces a new type-safe DSL for specifying columns when making a postgrest request. Example:

val columns = Columns {
    //Basic columns
    named("name", "id" withType "text") // name, id::text
    //Foreign columns
    foreign("actors") { // actors(name,total:count(),salary.avg())
        named("name")
        named(count() withAlias "total", "salary" withFunction avg())
    }
    foreign("studios" withAlias "studio") { // studio:studios(name)
        named("name")
    }
    //Or spread within the response
    foreign("studios") { // ...studios(studio_name:name)
        spread = true
        named("name" withAlias "studio_name")
        foreign("nested") {} // also possible
    }
    //JSON columns
    json("json_data", "key") // jsonData->key
    json("json_data", "list", "0", returnAsText = true) // jsonData->list->>0
}
supabase.from("movies").select(columns)

[github-advanced-security]

detekt found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.

[jan-tennert]

Not sure about the names here, maybe applyFunction or useFunction and castAs would be better

[grdsdev]

Hey Jan, that is an awesome feature.

I was thinking of improving this also on the Swift library, for Swift building a DSL as that is also possible, but I was thinking of another possibility, which is using Macro (I think that would be a custom annotation on Kotlin).

So we could have the columns definition also be the model returned by the request, such as:

@Selectable
struct Movie {
  let id: String
  let name: String

  @Foreign(table: "studios")
  let studio: Studio

  // Still need to think about how to express the other cases
}

Then on Swift we could use it as:

let movies: [Movie] = try await supabase.from("movies").select(Movie.self).execute.value

This was my initial thought, what do you think of it?

[jan-tennert]

That is also a great idea! Upon quick thinking, something like this could be done via KSP.
So similarly as with your example, you have a model which defines the relations and types:

@Selectable
@Serializable
data class Movie(
    val id: Int, //no ColumnName annotation means the property name will be used

    @ColumnName(name = "movie_title")
    val title: String, //"title" is basically the alias and "movie_title" is the actual column name

    @Foreign(table = "studios")
    val studio: Studio,

    @ApplyFunction(function = AggregateFunction.AVG)
    @ColumnName("column_name")
    val someAverage: Double, //apply "avg()" on "column_name" and use "someAverage" as the alias

    @JsonPath("json_data", "array", "0", "key", returnAsText = true) // jsonData->array->0->>key
    val jsonKey: String
)

And the compiler generates a MoveColumns property / object which you can use when selecting:

val movie: Movie = supabase.from("movies").select(MovieColumns).decodeSingle()

For casts, we could do some automatic casts, e.g. if the property type is a String, the column value gets cast to a text. For custom types, there could be something like a @ColumnTransformer(transformer = MyCustomTypeTransformer) annotation.

[grdsdev]

Yeah, you got it.

Would it be possible to call select with the Movie itself, instead of having another type MovieColumn?

[jan-tennert]

Well, KSP can only generate code. Maybe we can get the generated type via the declared Movie type, so that everything happens in the background

[jan-tennert]

I'm going to try out how good this works with KSP, making this PR a draft for now.

[jan-tennert]

@grdsdev So I think it works pretty well, but as I said the limitation of KSP (in comparison to e.g. a Compiler Plugin) is that it can only generate code (a Compiler Plugin basically way more complex and not really documented well, KSP is recommended). What I did now is to force the creation of a companion object, so that the KSP compiler generates an extension property:

@Selectable
@Serializable
data class Movie(
    val id: Int,

    @ColumnName(name = "movie_title")
    val title: String,

    @Foreign
    @ColumnName("studios")
    val studio: Studio,

    @ApplyFunction(function = ApplyFunction.AVG)
    @ColumnName("column_name")
    val someAverage: Double, 

    @ColumnName("json_data")
    @JsonPath("array", "0", "key", returnAsText = true)
    val jsonKey: String
) {
    companion object // Required, also forced at compile-time
}

Usage:

val movies: List<Movie> = supabase.from("movies").select(Movie.columns).decodeList()

What do you think?

[grdsdev]

Can KSP add an implementation of an interface to the type?

If so, you can create a Selectable interface that defines the companion object, and then make an overload of the select method that accepts a Selectable type.

In Swift would be:

protocol Selectable {
  static var columns: String { get }
}

@Selectable
struct Movie {
    // columns...
}

// Macro generated code:
extension Movie: Selectable {
    static var columns: String {
       """
        id,
        title,
        studio:studios(\(Studio.columns))
       """
    }

Then select overload would be:

func select<S: Selectable>(_ select: S.Type) {
    self.select(S.columns)
}

[jan-tennert]

No, in Kotlin you cannot implement an interface for an external type. I had another idea:

• The Postgrest module provides an interface, something like a ColumnRegistry which has one method:

fun getColumnsFor(type: KType): String

• The KSP compiler generates columns for annotated data classes, generates a SupabasePlugin which implements this interface and stores all the columns within.
• Devs now only need to install this plugin once in the SupabaseClientBuilder:

val supabase = createSupabaseClient(url, key) {
    install(Postgrest)
    install(PostgrestColumnRegistry) //this plugin is generated by the KSP compiler
}

And for selecting, we just add a new generic variant for select:

supabase.from("movies").select<Movie>()

behind the scenes, Postgrest will get the columns via the installed registry plugin.
This would remove the requirement for a companion object and would only need a single line of additional code.

I don't think there is any way to completely make this process invisible (with KSP and without reflection, due to MP support).

[jan-tennert]

Alternatively, the KSP compiler could also generate an extension function for the Postgrest config (instead of a whole plugin):

install(Postgrest) {
    addSelectableTypes()
}

But the rest would be the same

[grdsdev]

extension function on Postgres config is better I think, not a fan of having to install a plugin for this

[jan-tennert]

Alright, going to implement and PR it, there we can discuss annotation names etc.

[jan-tennert]

An early draft can be seen in #769. I'm still not sure how we should handle spreading embedded resources.

[grdsdev]

Awesome @jan-tennert will review it. I think we can just mark spreading as unsupported for now, not sure how we could support it, in Swift I think is possible due the macro system being a bit more powerful, but I need to make a few tests first.