Giter VIP home page Giter VIP logo

kysely-ctl's People

Contributors

dependabot[bot] avatar igalklebanov avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

kysely-ctl's Issues

Potential incompatibility with a supabase hosted project

Hi. First, thanks for creating kysely, the type inference has been awesome!


There's this issue I've been facing when using both of these at the same time:

  • a supabase hosted project
  • kysely-ctl migrations

The issue only happens when those two are used together. In isolation, the issue does not happen. In other words:

  • a hosted supabase project works fine with vanilla sql migrations
  • kysely-ctl migrations work fine with a vanilla postgres instance (it also works fine with the local supabase stack pnpm supabase start)

One crucial thing I should point out is that supabase hosted projects do not grant superuser privileges. Would you happen to know if that could be a cause of incompatibility between the two? Could it be possible that kysely tries to alter some default permissions or entities managed by Supabase?

JS Migrations Support

Hello! ๐Ÿ‘‹ Thank you for this project!

We're currently writing migrations in JS (no translation needed to use it in the SvelteKit server hook).

By default, kysely-ctl skips JS files: WARN Ignoring 2024-07-01.js - not a TS file.

By removing these lines, I could make it work:

if (!isTSFile(fileName)) {
consola.warn(`Ignoring \`${fileName}\` - not a TS file.`)
continue
}

Would it be possible to relax this requirement, so TS and JS are both supported?

"e.split is not a function or its return value is not iterable" when running seeds

I have a folder in app/database/seeds with a file called dev.ts with this content:

import { convert, Instant } from "@js-joda/core";
import type { Kysely } from "kysely";
import { newId } from "../../api/shared/id";
import { DB } from "../../database/types";

export async function seed(db: Kysely<DB>): Promise<void> {
  await db.insertInto("users").values({
    id: newId(),
    firstName: "Jack",
    lastName: "Shephard",
    email: "[email protected]",
    passwordHash: "...",
    signupDate: new Date(),
  }).execute();
}

This is my configuration:

import { defineConfig } from "kysely-ctl";
import { getDialect } from "./app/database";
import { CamelCasePlugin } from "kysely";

export default defineConfig({
  dialect: getDialect(),
  plugins: [new CamelCasePlugin()],
  migrations: {
    migrationFolder: "app/database/migrations",
  },
  seeds: {
    seedFolder: "app/database/seeds",
  },
});

when I run kysely seed run I get:

> kysely seed run

โ— Starting seed run                                                                                                                                       

 ERROR  e.split is not a function or its return value is not iterable                                                                                     

  at mapTsExtensions (node_modules/.pnpm/[email protected]/node_modules/tsx/dist/register-Bx9fr_2c.mjs:1:620)
  at tryTsPaths (node_modules/.pnpm/[email protected]/node_modules/tsx/dist/esm/index.mjs?1720187035558:2:3222)
  at resolve (node_modules/.pnpm/[email protected]/node_modules/tsx/dist/esm/index.mjs?1720187035558:2:4233)
  at async nextResolve (node:internal/modules/esm/hooks:832:22)
  at async Hooks.resolve (node:internal/modules/esm/hooks:277:24)
  at async handleMessage (node:internal/modules/esm/worker:168:18) 



 ERROR  e.split is not a function or its return value is not iterable    

What am I doing wrong?

Thanks in advance ๐Ÿ™

defineConfig using --environment type issue with seedFolder

Error: "Type 'string' is not assignable to type 'undefined'.ts(2322)" seen when doing:

export default defineConfig({
  dialect: dialect,
  plugins: plugins,
  migrations: {
    migrationFolder: "./src/db/migrations",
  },
  $development: {
    seeds: {
      // @ts-expect-error for now igal on discord said he is looking into it
      seedFolder: "./src/db/seeds/development",
    },
  },
});

image

Additionally it also only lets you use the environment name strings found in c12's C12InputConfig interface but kysely-ctl's src/arguments/environment.mts has valueHint of 'prod | dev | test | ...' but you actually can't use those abbreviations for production and development without other typescript compilation error.

Object literal may only specify known properties, and '$dev' does not exist in type '(C12InputConfig<KyselyCTLConfig, ConfigLayerMeta> & KyselyCTLConfigBase & { ...; } & { ...; } & { ...; }) | ... 7 more ... | (C12InputConfig<...> & ... 3 more ... & { ...; })'.ts(2353)

image

Per @igalklebanov review on discord issue is appropriate to raise on this.

Running `migrate` commands causes package manager to install

Notice that when running migrate:latest, as an example (but this also happens with most of the other migrate commands, it causes my package manager (pnpm in this case) to run an install, which significantly slows down the operation. Is this expected behavior?

โžœ  pnpm kysely migrate:latest           
> dotenv -e .env.local -- kysely "migrate:latest"

../../..                                 | Progress: resolved 796, reused 718, downloaded 0, added 0, done
Done in 2.5s
โ— Starting migration to latest                                                                                                                      
โ„น Migration skipped: no new migrations found

Use a unique db instance to perform scripts.

  • could we allow the defineConfig to accept the same db instance to perform scripts?

given a project where you have a db instance definition pointing to your desired dialect like usual,
we would have something like:

import { defineConfig } from 'kysely-ctl'

export default defineConfig({
  db,
  migrations: {
     migrationFolder: path.join(__dirname, './src/db/migrations')
  },
})

Kysely already provides a clear and simple Migrator class that is not related to any extra configuration or dependency.
Hopefully think this could make a bit easier to use like the migration-cli.

ERROR: Could not find the version of the CLI

It appears the CLI is looking at the version of my application package (which is in a monorepo, using pnpm workspaces, and is not versioned) rather than the version of kysely-ctl.

 ERROR  Could not find the version of the CLI

  at getCLIInstalledVersion (/PROJECT/node_modules/.pnpm/[email protected][email protected][email protected][email protected][email protected][email protected][email protected]_pk_2dkvxtov55pjiwaee5hspcr3bm/node_modules/kysely-ctl/dist/bin.js:77:11)

Support `Kysely<Database>` @ `seed make`.

Hey ๐Ÿ‘‹

Currently kysely seed make uses a template with Kysely<any>. Seed files, unlike migrations, should evolve over time with the latest schema. They should be type-checked and maintained.

Need to find a way of supporting a type-safe template.

Migration fails with: "TypeError: Cannot redefine property: then"

I created the following migration file:

import { Kysely, sql } from 'kysely'

export async function up(db: Kysely<any>): Promise<void> {
  await db.schema
    .createTable('person')
    .addColumn('id', 'serial', (col) => col.primaryKey())
    .addColumn('first_name', 'varchar', (col) => col.notNull())
    .addColumn('last_name', 'varchar')
    .addColumn('gender', 'varchar(50)', (col) => col.notNull())
    .addColumn('created_at', 'timestamp', (col) =>
      col.defaultTo(sql`now()`).notNull()
    )
    .execute()
}

export async function down(db: Kysely<any>): Promise<void> {
  await db.schema.dropTable('person').execute()
}

And then I tried to run the migration with npm run kysely migrate:up but it failed with the following error message:

Migration failed with TypeError: Cannot redefine property: then

If I remove the following code and retry the migration, it completes successfully:

.addColumn('created_at', 'timestamp', (col) =>
  col.defaultTo(sql`now()`).notNull()
)

In my kysely.config.ts file I have the following code:

import { Kysely, PostgresDialect } from "kysely";
import { defineConfig } from "kysely-ctl";
import { Pool } from 'pg'

const connectionString = process.env.DATABASE_URL ?? ''

const dialect = new PostgresDialect({
  pool: new Pool({
    connectionString: connectionString
  })
})

export default defineConfig({
  dialect: dialect,
  migrations: {
    migrationFolder: 'migrations',
  },
});

It seems a bug to me but I'm not sure if I'm missing something else.

How to use the "seeds" feature?

There doesn't seem to be any mention of what the seed files are supposed to contain. Is there maybe some documentation I'm missing where it explains what the structure of these files are, how to async/await in these files or how to connect to the database (e.g. it's passed in like for migrations vs we should import the connection).

Thanks in advance and thanks for this awesome library!

Cannot find module 'bun:sqlite'

Reproduction Steps:

Clone this repository, and cd into the examples/bun folder. Attempt to run bun kysely migrate:latest.

Receive Error:

 ERROR  Cannot find module 'bun:sqlite'
Require stack:
- /kysely-ctl/examples/bun/.config/kysely.config.ts

  Require stack:
  - .config/kysely.config.ts
  at Module._resolveFilename (node:internal/modules/cjs/loader:1145:15)
  at Function.resolve (node:internal/modules/helpers:190:19)
  at _resolve (node_modules/jiti/dist/jiti.js:1:241814)
  at jiti (node_modules/jiti/dist/jiti.js:1:244531)
  at .config/kysely.config.ts:3:18
  at evalModule (node_modules/jiti/dist/jiti.js:1:247313)
  at Object.jiti (node_modules/jiti/dist/jiti.js:1:245241)
  at resolveConfig (node_modules/c12/dist/shared/c12.cab0c9da.mjs:345:26)
  at loadConfig (node_modules/c12/dist/shared/c12.cab0c9da.mjs:147:29)
  at async getConfig (node_modules/kysely-ctl/dist/bin.js:215:24)
  at async getConfigOrFail (node_modules/kysely-ctl/dist/bin.js:247:18)
  at async usingMigrator (node_modules/kysely-ctl/dist/bin.js:470:18)
  at async Object.run (node_modules/kysely-ctl/dist/bin.js:540:5)
  at async runCommand (node_modules/citty/dist/index.mjs:316:16)
  at async runCommand (node_modules/citty/dist/index.mjs:307:11)
  at async runMain (node_modules/citty/dist/index.mjs:445:7)
  at async Object.parse (node_modules/kysely-ctl/dist/bin.js:960:7)

 ERROR  Cannot find module 'bun:sqlite'

required to install 'ws' to perform migrations from 'neondb'

I'm required to install the 'ws' dependency into my project in order to perform migration on an external 'neondb' connection

error message:

> kysely migrate:down

โ— Starting migration down
โœ– Migration failed with Error: All attempts to open a WebSocket to connect to the database failed. Please refer to https://github.com/neondatabase/serverless/blob/main/CONFIG.md#websocketconstructor-typeof-websocket--undefined. Details: fetch failed

temporary fix:

import { Pool, neonConfig } from '@neondatabase/serverless'
import ws from 'ws'

neonConfig.webSocketConstructor = ws

const pool = new Pool({ connectionString: process.env.DB_ADDRESS })

export default defineConfig({
  dialect: new PostgresDialect({ pool }),
  migrations: {
    migrationFolder: './src/lib/db/migrations',
  },
})

expected result:
not having to install an ws dependency since the connection to run the project works just fine:

import { Kysely, PostgresDialect } from 'kysely'

const pool = new Pool({ connectionString: process.env.DB_ADDRESS })

export const db = new Kysely<...>({
  dialect: new PostgresDialect({ pool }),
})

running with previous migration-cli would work fine on this scenario.

we might be able to fix that by updating conditions on getDialect?

Why suggesting peer dependencies

Hi, I'm so glad that you are adopting unjs projects โค๏ธ

I noticed something a little strange about usage though since you are advising to install needed depes as peer dependencies. May I ask the reasons behind it?


PS: Note sure if you also know about jiti project. Aside from top-level await support I like to know what were main limitations if you ever considered it (this dependency at least comes out of the box by c12 so at least will make install smaller)

Allow absolute path for migrationFolder config

For the following configuration

export default defineConfig({
  kysely: kyselyInstance,
  migrations: {
    migrationFolder: path.resolve(__dirname, './path/to/migrations'),
  },
});

Getting the error

Migration failed with Error: ENOENT: no such file or directory, mkdir '/Users/.../project/Users/.../project/path/to/migrations'

Seems like it concatenates cwd with provided path and makes it impossible to pass absolute path.

--

It might be non-intuitive to figure out you need to pass path relative to the directory (cwd) from where you execute the command. Would be easier to follow if one can just pass absolute path.

Seed fails, yet cli returns with exit code 0

Current: kysely seed run exits with exit code 0 when the seed operation encounters an error (such as unique constraint violation).

Expected: kysely seed run exits with exit code 1 when the seed operation encounters an error


Example output where the seed fails, yet the cli command returns with exit code 0:

$ pnpm kysely seed run

โš™ { rawArgs: [],
  args: { _: [], debug: false },
  data: undefined,
  cmd:
   { meta: { description: 'Run seed files', name: 'run' },
     args:
      { cwd: [Object],
        debug: [Object],
        environment: [Object],
        'no-outdated-check': [Object],
        specific: [Object] },
     run: [AsyncFunction: run] } } []
โ— Starting seed run
โš™ { config:
   { dialect: PostgresDialect {},
     migrations: { migrationFolder: 'src/migrations' },
     seeds: { seedFolder: 'src/seeds' } },
โš™ { error:
   { error: duplicate key value violates unique constraint "foo_bar_id_key"
       at node_modules/pg/lib/client.js:526:17
     length: 217,
     severity: 'ERROR',
     code: '23505',
     detail: 'Key (bar_id)=(1) already exists.',
     hint: undefined,
     position: undefined,
     internalPosition: undefined,
     internalQuery: undefined,
     where: undefined,
     schema: 'public',
     table: 'foo',
     column: undefined,
     dataType: undefined,
     constraint: 'foo_bar_id_key',
     file: 'nbtinsert.c',
     line: '664',
     routine: '_bt_check_unique' },
  results: [ { seedName: '1719484759291_init', status: 'Error' } ] }
โ„น Ran 1 seed:
[โœ—] 1719484759291_init - error: duplicate key value violates unique constraint "foo_bar_id_key"

Migration error message do not include any details on connection error

When trying to run migrations today I received the following error:

kysely migrate up   
โ— Starting migration up                                                                                                                                                                                                                       
โœ– Migration failed with AggregateError   

This provides no information that could help with debugging the issue. Manually adding console.log(error) directly above the current Migration failed with logging code, reveals the following details.

kysely migrate up   
โ— Starting migration up                                                                      
AggregateError
    at internalConnectMultiple (node:net:1114:18)
    at afterConnectMultiple (node:net:1667:5) {
  code: 'ECONNREFUSED',
  fatal: true,
  [errors]: [
    Error: connect ECONNREFUSED ::1:3306
        at createConnectionError (node:net:1634:14)
        at afterConnectMultiple (node:net:1664:40) {
      errno: -61,
      code: 'ECONNREFUSED',
      syscall: 'connect',
      address: '::1',
      port: 3306
    },
    Error: connect ECONNREFUSED 127.0.0.1:3306
        at createConnectionError (node:net:1634:14)
        at afterConnectMultiple (node:net:1664:40) {
      errno: -61,
      code: 'ECONNREFUSED',
      syscall: 'connect',
      address: '127.0.0.1',
      port: 3306
    }
  ]
}
โœ– Migration failed with AggregateError

So the kysely-ctl has access to all the details, but the current error message parsing - basically calling toString on the object by putting it as a part of the string template - is not sufficient to show them.

Not sure what is the best solution here, since errors can come from various libraries depending on the connection type. However, I've seen the same error message when running migrations on a postgres database, so it's definitely not limited just to mysql connection. I haven't had this issue with other error types, so maybe checking if an error that is thrown is an AggregateError, and adding a custom formatting for it, would be sufficient?

allow stub templates

I have seen in other migration cli's the use of stubs which are really nice.

I am thinking a command like
npx kysely migrate:stub {stub_name} {migration_name}

Would need to add a stubsFolder option under migrations option in the config file

Then I could have a folder called whatever I want with file names like: my_stub_file.stub. This file would be just like your template file in your src->templates folder.

When the migration is created it will use the stub file as the template, name the file as it does now and use the correct extension.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.