sqiffy (or just squiffy πΉ) - Experimental compound SQL framework with type-safe DSL API generated at compile-time from scheme diff. It is dedicated for applications, plugins & libraries responsible for internal database management.
Table of contents:
- User defines versioned table definition using
@Defintion
annotation - Sqiffy's annotation processor (KSP) at compile-time:
- Converts table definitions into versioned changelog, similar to Liquibase
- Generates up-to-date entity data classes for Kotlin with KotlinPoet
- Creates bindings for Exposed (DSL) framework
- Validates schemes and bindings to eliminate typos and invalid operations
- When application starts, you can run set of prepared versioned migrations against current database state
Database | Support | Notes |
---|---|---|
PostgreSQL, Embedded PostgreSQL | Full support | Main target of the library. |
MariaDB, MySQL | Supported | All operations should be supported, but some of the features might not be available. |
SQLite | Work in progress | SQLite does not provide several crucial schema update queries & type system is flexible. Because of that, schema updates are based on top of the modifications applied to sqlite_master , but the stability of this solution is unknown. See #2 for more details. |
H2 (MySQL mode) | Unstable | Such as SQLite, H2 implements SQL standard on their own & some of the compatibility features are just a fake mocks. In most cases, it's just better to use other databases (or their embedded variants). |
Gradle (kts):
plugins {
id("com.google.devtools.ksp") version "1.9.22-1.0.17" // for Kotlin 1.9.22
}
dependencies {
val sqiffy = "1.0.0-alpha.64"
ksp("com.dzikoysk.sqiffy:sqiffy-symbol-processor:$sqiffy") // annotation processor
implementation("com.dzikoysk.sqiffy:sqiffy:$sqiffy") // core library & implementation
}
Describe your table using versioned definitions:
object UserAndGuildScenarioVersions {
const val V_1_0_0 = "1.0.0"
const val V_1_0_1 = "1.0.1"
const val V_1_0_2 = "1.0.2"
}
@EnumDefinition(name = "role", mappedTo = "Role", [
EnumVersion(
version = V_1_0_0,
operation = ADD_VALUES,
values = ["ADMIN", "USER"]
),
EnumVersion(
version = V_1_0_1,
operation = ADD_VALUES,
values = ["MODERATOR", "SPECTATOR"]
)
])
object RoleDefinition
@Definition([
DefinitionVersion(
version = V_1_0_0,
name = "users_table",
properties = [
Property(name = "id", type = SERIAL),
Property(name = "uuid", type = UUID_TYPE),
Property(name = "name", type = VARCHAR, details = "12"),
Property(name = "role", type = ENUM, enumDefinition = RoleDefinition::class)
],
constraints = [
Constraint(type = PRIMARY_KEY, name = "pk_id", on = ["id"]),
],
indices = [
Index(type = INDEX, name = "idx_id", columns = ["id"]),
Index(type = UNIQUE_INDEX, name = "uq_name", columns = ["name"])
]
),
DefinitionVersion(
version = V_1_0_1,
properties = [
Property(operation = RETYPE, name = "name", type = VARCHAR, details = "24"),
Property(operation = ADD, name = "display_name", type = VARCHAR, details = "48", nullable = true),
],
indices = [
Index(operation = REMOVE_INDEX, type = INDEX, name = "idx_id"),
Index(type = INDEX, name = "idx_id", columns = ["id"])
]
),
DefinitionVersion(
version = V_1_0_2,
properties = [
Property(operation = RENAME, name = "display_name", rename = "displayName")
]
)
])
object UserDefinition
@Definition([
DefinitionVersion(
version = V_1_0_0,
name = "guilds_table",
properties = [
Property(name = "id", type = SERIAL),
Property(name = "name", type = VARCHAR, details = "24"),
Property(name = "owner", type = INT)
],
constraints = [
Constraint(type = FOREIGN_KEY, on = ["id"], name = "fk_id", referenced = UserDefinition::class, references = "id")
]
),
DefinitionVersion(
version = V_1_0_1,
constraints = [
Constraint(REMOVE_CONSTRAINT, type = FOREIGN_KEY, name = "fk_id")
]
),
DefinitionVersion(
version = V_1_0_2,
constraints = [
Constraint(type = FOREIGN_KEY, on = ["id"], name = "fk_id", referenced = UserDefinition::class, references = "id")
]
)
])
object GuildDefinition
Build your project, so KSP can generate classes on top of the specified changelog. In this case it'll generate:
User
,Guild
data classUnidentifiedUser
,UnidentifiedGuild
data class without autogenerated keys (like e.g. serial id)UserTableNames
,GuildTableNames
object with table & column namesRole
enum that is based on linked in scheme@EnumDefinition
UserTable
,GuildTable
implementation ofTable
object for built-in DSL- SQL migrations between each version
Then, you can simply connect to the database & run migrations:
this.database = Sqiffy.createDatabase(
dataSource = createDataSource(),
logger = Slf4JSqiffyLogger(LoggerFactory.getLogger(SqiffyDatabase::class.java))
)
val changeLog = database.generateChangeLog(UserDefinition::class, GuildDefinition::class)
database.runMigrations(changeLog = changeLog)
// [..] use database
database.close()
You can also execute queries using generated DSL:
val userToInsert = UnidentifiedUser(
name = "Panda",
displayName = "Only Panda",
uuid = UUID.randomUUID(),
role = Role.MODERATOR
)
val insertedUserWithDsl = database
.insert(UserTable) {
it[UserTable.uuid] = userToInsert.uuid
it[UserTable.name] = userToInsert.name
it[UserTable.displayName] = userToInsert.displayName
it[UserTable.role] = userToInsert.role
}
.map { userToInsert.withId(id = it[UserTable.id]) }
.first()
val guildToInsert = UnidentifiedGuild(
name = "MONKE",
owner = insertedUserWithDsl.id
)
val insertedGuild = database
.insert(GuildTable) {
it[GuildTable.name] = guildToInsert.name
it[GuildTable.owner] = guildToInsert.owner
}
.map { guildToInsert.withId(id = it[GuildTable.id]) }
.first()
println("Inserted user: $insertedUserWithDsl")
val userFromDatabaseUsingDsl = database.select(UserTable)
.where { UserTable.uuid eq insertedUserWithDsl.uuid }
.map {
User(
id = it[UserTable.id],
name = it[UserTable.name],
uuid = it[UserTable.uuid],
displayName = it[UserTable.displayName],
role = it[UserTable.role]
)
}
.firstOrNull()
println("Loaded user: $userFromDatabaseUsingDsl")
val joinedData = database.select(UserTable)
.join(INNER, UserTable.id, GuildTable.owner)
.where { GuildTable.owner eq insertedGuild.owner }
.map { it[UserTable.name] to it[GuildTable.name] }
.first()
println(joinedData)
Or you can use generated names to execute manually, using e.g. JDBI:
val userFromDatabaseUsingRawJdbi = database.getJdbi().withHandle<User, Exception> { handle ->
handle
.select(multiline("""
SELECT *
FROM "${UserTableNames.TABLE}"
WHERE "${UserTableNames.NAME}" = :nameToMatch
"""))
.bind("nameToMatch", "Panda")
.mapTo<User>()
.firstOrNull()
}
println("Loaded user: $userFromDatabaseUsingRawJdbi")
The comparison shows differences between multiple approaches to database management, there's no "best" approach, it's all about your preferences and needs. Sqiffy combines some known mechanisms to address issues of other approaches within the ecosystem of bundled applications shared among multiple users.
Approach | Easy to use | Control over the schema | One source of truth | Multiple dialects | Auto-migrations | Type-safe | Compile-time validation | DSL |
---|---|---|---|---|---|---|---|---|
Raw
You want to avoid complex libraries and use raw SQL. Because of that, it increases amount of code you have to write, and it's error-prone. |
β | β | β | β | β | β | β | β |
SQL wrapper
Libraries like JDBI simplifies interaction with raw SQL and entities, but there's still much to do around it manually. |
β | β | β | β | β | β | β | Β½ |
ORM
ORM libraries promise to handle all the database stuff for you, but you're party losing control over the implementation and it may turn against you. |
β | β | β | β | Β½ | Β½ | β | Β½ |
DSL
Libraries like Exposed DSL provides very convenient type-safe API and basic scheme generators, but you partly lose control over the schema and you may encounter several issues with their API that doesn't cover all your needs. |
β | β | β | Β½ | Β½ | Β½ | Β½ | β |
DSL/ORM + Liquibase/Flyway
Migrations are a must-have for any database management system, but it's not easy to implement them in a type-safe way and implement them for multiple dialects & users. |
Β½ | β | β | Β½ | β | Β½ | β | N/A |
JOOQ
JOOQ defines a new category, and while it's pretty good escape from regular DSL and uncontrolled ORMs, it targets enterprise products with an existing databases controlled by 3rd party sources, so it's not that good for any kind of bundled application. |
β | N/A | β | Β½ | β | β | β | β |
Sqiffy
Combines several features mentioned above as opt-in and handles bundled database schema changelog. |
β | β | β | β | β | β | β | β |
β - Yes
β - No
Β½ - Partially or not exactly matching our target (bundled apps with swappable database & dialects)
N/A - Not applicable or given library is not responsible for this feature