kysely-org / kysely-ctl Goto Github PK
View Code? Open in Web Editor NEWCommand-line tool for Kysely
License: MIT License
Command-line tool for Kysely
License: MIT License
Hi. First, thanks for creating kysely, the type inference has been awesome!
There's this issue I've been facing when using both of these at the same time:
The issue only happens when those two are used together. In isolation, the issue does not happen. In other words:
pnpm supabase start
)One crucial thing I should point out is that supabase hosted projects do not grant superuser privileges. Would you happen to know if that could be a cause of incompatibility between the two? Could it be possible that kysely tries to alter some default permissions or entities managed by Supabase?
Hello! ๐ Thank you for this project!
We're currently writing migrations in JS (no translation needed to use it in the SvelteKit server hook).
By default, kysely-ctl
skips JS files: WARN Ignoring 2024-07-01.js - not a TS file.
By removing these lines, I could make it work:
kysely-ctl/src/kysely/ts-file-migration-provider.mts
Lines 27 to 30 in 230798d
Would it be possible to relax this requirement, so TS and JS are both supported?
I have a folder in app/database/seeds
with a file called dev.ts
with this content:
import { convert, Instant } from "@js-joda/core";
import type { Kysely } from "kysely";
import { newId } from "../../api/shared/id";
import { DB } from "../../database/types";
export async function seed(db: Kysely<DB>): Promise<void> {
await db.insertInto("users").values({
id: newId(),
firstName: "Jack",
lastName: "Shephard",
email: "[email protected]",
passwordHash: "...",
signupDate: new Date(),
}).execute();
}
This is my configuration:
import { defineConfig } from "kysely-ctl";
import { getDialect } from "./app/database";
import { CamelCasePlugin } from "kysely";
export default defineConfig({
dialect: getDialect(),
plugins: [new CamelCasePlugin()],
migrations: {
migrationFolder: "app/database/migrations",
},
seeds: {
seedFolder: "app/database/seeds",
},
});
when I run kysely seed run
I get:
> kysely seed run
โ Starting seed run
ERROR e.split is not a function or its return value is not iterable
at mapTsExtensions (node_modules/.pnpm/[email protected]/node_modules/tsx/dist/register-Bx9fr_2c.mjs:1:620)
at tryTsPaths (node_modules/.pnpm/[email protected]/node_modules/tsx/dist/esm/index.mjs?1720187035558:2:3222)
at resolve (node_modules/.pnpm/[email protected]/node_modules/tsx/dist/esm/index.mjs?1720187035558:2:4233)
at async nextResolve (node:internal/modules/esm/hooks:832:22)
at async Hooks.resolve (node:internal/modules/esm/hooks:277:24)
at async handleMessage (node:internal/modules/esm/worker:168:18)
ERROR e.split is not a function or its return value is not iterable
What am I doing wrong?
Thanks in advance ๐
Error: "Type 'string' is not assignable to type 'undefined'.ts(2322)" seen when doing:
export default defineConfig({
dialect: dialect,
plugins: plugins,
migrations: {
migrationFolder: "./src/db/migrations",
},
$development: {
seeds: {
// @ts-expect-error for now igal on discord said he is looking into it
seedFolder: "./src/db/seeds/development",
},
},
});
Additionally it also only lets you use the environment name strings found in c12's C12InputConfig interface but kysely-ctl's src/arguments/environment.mts has valueHint of 'prod | dev | test | ...' but you actually can't use those abbreviations for production and development without other typescript compilation error.
Object literal may only specify known properties, and '$dev' does not exist in type '(C12InputConfig<KyselyCTLConfig, ConfigLayerMeta> & KyselyCTLConfigBase & { ...; } & { ...; } & { ...; }) | ... 7 more ... | (C12InputConfig<...> & ... 3 more ... & { ...; })'.ts(2353)
Per @igalklebanov review on discord issue is appropriate to raise on this.
Hey ๐
What if we could dry-run migrations?
Hey ๐
Changing defineConfig
's input type can easily break dialect
prop autocompletion in your IDE. I nearly shipped #48 with no autocompletion support.
Gotta find a way to test this still works in CI.
Notice that when running migrate:latest
, as an example (but this also happens with most of the other migrate
commands, it causes my package manager (pnpm
in this case) to run an install, which significantly slows down the operation. Is this expected behavior?
โ pnpm kysely migrate:latest
> dotenv -e .env.local -- kysely "migrate:latest"
../../.. | Progress: resolved 796, reused 718, downloaded 0, added 0, done
Done in 2.5s
โ Starting migration to latest
โน Migration skipped: no new migrations found
given a project where you have a db instance definition pointing to your desired dialect like usual,
we would have something like:
import { defineConfig } from 'kysely-ctl'
export default defineConfig({
db,
migrations: {
migrationFolder: path.join(__dirname, './src/db/migrations')
},
})
Kysely already provides a clear and simple Migrator class that is not related to any extra configuration or dependency.
Hopefully think this could make a bit easier to use like the migration-cli.
It appears the CLI is looking at the version of my application package (which is in a monorepo, using pnpm workspaces, and is not versioned) rather than the version of kysely-ctl
.
ERROR Could not find the version of the CLI
at getCLIInstalledVersion (/PROJECT/node_modules/.pnpm/[email protected][email protected][email protected][email protected][email protected][email protected][email protected]_pk_2dkvxtov55pjiwaee5hspcr3bm/node_modules/kysely-ctl/dist/bin.js:77:11)
Hey ๐
Currently kysely seed make
uses a template with Kysely<any>
. Seed files, unlike migrations, should evolve over time with the latest schema. They should be type-checked and maintained.
Need to find a way of supporting a type-safe template.
I created the following migration file:
import { Kysely, sql } from 'kysely'
export async function up(db: Kysely<any>): Promise<void> {
await db.schema
.createTable('person')
.addColumn('id', 'serial', (col) => col.primaryKey())
.addColumn('first_name', 'varchar', (col) => col.notNull())
.addColumn('last_name', 'varchar')
.addColumn('gender', 'varchar(50)', (col) => col.notNull())
.addColumn('created_at', 'timestamp', (col) =>
col.defaultTo(sql`now()`).notNull()
)
.execute()
}
export async function down(db: Kysely<any>): Promise<void> {
await db.schema.dropTable('person').execute()
}
And then I tried to run the migration with npm run kysely migrate:up
but it failed with the following error message:
Migration failed with TypeError: Cannot redefine property: then
If I remove the following code and retry the migration, it completes successfully:
.addColumn('created_at', 'timestamp', (col) =>
col.defaultTo(sql`now()`).notNull()
)
In my kysely.config.ts file I have the following code:
import { Kysely, PostgresDialect } from "kysely";
import { defineConfig } from "kysely-ctl";
import { Pool } from 'pg'
const connectionString = process.env.DATABASE_URL ?? ''
const dialect = new PostgresDialect({
pool: new Pool({
connectionString: connectionString
})
})
export default defineConfig({
dialect: dialect,
migrations: {
migrationFolder: 'migrations',
},
});
It seems a bug to me but I'm not sure if I'm missing something else.
There doesn't seem to be any mention of what the seed files are supposed to contain. Is there maybe some documentation I'm missing where it explains what the structure of these files are, how to async
/await
in these files or how to connect to the database (e.g. it's passed in like for migrations vs we should import the connection).
Thanks in advance and thanks for this awesome library!
Reproduction Steps:
Clone this repository, and cd into the examples/bun
folder. Attempt to run bun kysely migrate:latest
.
Receive Error:
ERROR Cannot find module 'bun:sqlite'
Require stack:
- /kysely-ctl/examples/bun/.config/kysely.config.ts
Require stack:
- .config/kysely.config.ts
at Module._resolveFilename (node:internal/modules/cjs/loader:1145:15)
at Function.resolve (node:internal/modules/helpers:190:19)
at _resolve (node_modules/jiti/dist/jiti.js:1:241814)
at jiti (node_modules/jiti/dist/jiti.js:1:244531)
at .config/kysely.config.ts:3:18
at evalModule (node_modules/jiti/dist/jiti.js:1:247313)
at Object.jiti (node_modules/jiti/dist/jiti.js:1:245241)
at resolveConfig (node_modules/c12/dist/shared/c12.cab0c9da.mjs:345:26)
at loadConfig (node_modules/c12/dist/shared/c12.cab0c9da.mjs:147:29)
at async getConfig (node_modules/kysely-ctl/dist/bin.js:215:24)
at async getConfigOrFail (node_modules/kysely-ctl/dist/bin.js:247:18)
at async usingMigrator (node_modules/kysely-ctl/dist/bin.js:470:18)
at async Object.run (node_modules/kysely-ctl/dist/bin.js:540:5)
at async runCommand (node_modules/citty/dist/index.mjs:316:16)
at async runCommand (node_modules/citty/dist/index.mjs:307:11)
at async runMain (node_modules/citty/dist/index.mjs:445:7)
at async Object.parse (node_modules/kysely-ctl/dist/bin.js:960:7)
ERROR Cannot find module 'bun:sqlite'
Hey ๐
Users are unable to do the connection between kysely-ctl
and c12
and figure out how to use --environment
given the current documentation available on kysely-ctl
's side.
Need to refer to https://github.com/unjs/c12#environment-specific-configuration in -h
/--help
text.
I'm required to install the 'ws' dependency into my project in order to perform migration on an external 'neondb' connection
error message:
> kysely migrate:down
โ Starting migration down
โ Migration failed with Error: All attempts to open a WebSocket to connect to the database failed. Please refer to https://github.com/neondatabase/serverless/blob/main/CONFIG.md#websocketconstructor-typeof-websocket--undefined. Details: fetch failed
temporary fix:
import { Pool, neonConfig } from '@neondatabase/serverless'
import ws from 'ws'
neonConfig.webSocketConstructor = ws
const pool = new Pool({ connectionString: process.env.DB_ADDRESS })
export default defineConfig({
dialect: new PostgresDialect({ pool }),
migrations: {
migrationFolder: './src/lib/db/migrations',
},
})
expected result:
not having to install an ws dependency since the connection to run the project works just fine:
import { Kysely, PostgresDialect } from 'kysely'
const pool = new Pool({ connectionString: process.env.DB_ADDRESS })
export const db = new Kysely<...>({
dialect: new PostgresDialect({ pool }),
})
running with previous migration-cli would work fine on this scenario.
we might be able to fix that by updating conditions on getDialect
?
Hi, I'm so glad that you are adopting unjs projects โค๏ธ
I noticed something a little strange about usage though since you are advising to install needed depes as peer dependencies. May I ask the reasons behind it?
PS: Note sure if you also know about jiti project. Aside from top-level await
support I like to know what were main limitations if you ever considered it (this dependency at least comes out of the box by c12 so at least will make install smaller)
For the following configuration
export default defineConfig({
kysely: kyselyInstance,
migrations: {
migrationFolder: path.resolve(__dirname, './path/to/migrations'),
},
});
Getting the error
Migration failed with Error: ENOENT: no such file or directory, mkdir '/Users/.../project/Users/.../project/path/to/migrations'
Seems like it concatenates cwd
with provided path and makes it impossible to pass absolute path.
--
It might be non-intuitive to figure out you need to pass path relative to the directory (cwd) from where you execute the command. Would be easier to follow if one can just pass absolute path.
Current: kysely seed run
exits with exit code 0 when the seed operation encounters an error (such as unique constraint violation).
Expected: kysely seed run
exits with exit code 1 when the seed operation encounters an error
Example output where the seed fails, yet the cli command returns with exit code 0:
$ pnpm kysely seed run
โ { rawArgs: [],
args: { _: [], debug: false },
data: undefined,
cmd:
{ meta: { description: 'Run seed files', name: 'run' },
args:
{ cwd: [Object],
debug: [Object],
environment: [Object],
'no-outdated-check': [Object],
specific: [Object] },
run: [AsyncFunction: run] } } []
โ Starting seed run
โ { config:
{ dialect: PostgresDialect {},
migrations: { migrationFolder: 'src/migrations' },
seeds: { seedFolder: 'src/seeds' } },
โ { error:
{ error: duplicate key value violates unique constraint "foo_bar_id_key"
at node_modules/pg/lib/client.js:526:17
length: 217,
severity: 'ERROR',
code: '23505',
detail: 'Key (bar_id)=(1) already exists.',
hint: undefined,
position: undefined,
internalPosition: undefined,
internalQuery: undefined,
where: undefined,
schema: 'public',
table: 'foo',
column: undefined,
dataType: undefined,
constraint: 'foo_bar_id_key',
file: 'nbtinsert.c',
line: '664',
routine: '_bt_check_unique' },
results: [ { seedName: '1719484759291_init', status: 'Error' } ] }
โน Ran 1 seed:
[โ] 1719484759291_init - error: duplicate key value violates unique constraint "foo_bar_id_key"
When trying to run migrations today I received the following error:
kysely migrate up
โ Starting migration up
โ Migration failed with AggregateError
This provides no information that could help with debugging the issue. Manually adding console.log(error)
directly above the current Migration failed with
logging code, reveals the following details.
kysely migrate up
โ Starting migration up
AggregateError
at internalConnectMultiple (node:net:1114:18)
at afterConnectMultiple (node:net:1667:5) {
code: 'ECONNREFUSED',
fatal: true,
[errors]: [
Error: connect ECONNREFUSED ::1:3306
at createConnectionError (node:net:1634:14)
at afterConnectMultiple (node:net:1664:40) {
errno: -61,
code: 'ECONNREFUSED',
syscall: 'connect',
address: '::1',
port: 3306
},
Error: connect ECONNREFUSED 127.0.0.1:3306
at createConnectionError (node:net:1634:14)
at afterConnectMultiple (node:net:1664:40) {
errno: -61,
code: 'ECONNREFUSED',
syscall: 'connect',
address: '127.0.0.1',
port: 3306
}
]
}
โ Migration failed with AggregateError
So the kysely-ctl has access to all the details, but the current error message parsing - basically calling toString on the object by putting it as a part of the string template - is not sufficient to show them.
Not sure what is the best solution here, since errors can come from various libraries depending on the connection type. However, I've seen the same error message when running migrations on a postgres database, so it's definitely not limited just to mysql connection. I haven't had this issue with other error types, so maybe checking if an error that is thrown is an AggregateError, and adding a custom formatting for it, would be sufficient?
I have seen in other migration cli's the use of stubs which are really nice.
I am thinking a command like
npx kysely migrate:stub {stub_name} {migration_name}
Would need to add a stubsFolder option under migrations option in the config file
Then I could have a folder called whatever I want with file names like: my_stub_file.stub
. This file would be just like your template file in your src->templates folder.
When the migration is created it will use the stub file as the template, name the file as it does now and use the correct extension.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.