The Database Migration Tool was designed for those looking to migrate their data from one database to another. Additionally type casting is supported if you need to change column type in new version. Our main goal is to make data migration possible between different database servers. For example mysql, postgresql.
Currently only mysql and postgresql is supported. Future planning is to support migration from/to other databases.
Documentation: Documentation
pip install madmigrationmadmigration -f migration_schema.yamlAfter installation you should define YAML file where configuration will be taken in order to apply data to target database. Yaml file with list and dictionaries may contain following structures:
- SourceConfig is intented to be data from source database
- DestinationConfig is intented to be transfered data to target database
version: 0.1.6
Configs:
- SourceConfig:
dbURI: "postgres://admin:[email protected]/oldDB"
- DestinationConfig:
dbURI: "mysql://admin:[email protected]/newDB"
migrationTables:
- migrationTable:
SourceTable:
name: users
DestinationTable:
name: persons
create: True
MigrationColumns:
- sourceColumn:
name: id
destinationColumn:
name: id
options:
type_cast: bigint
primary_key: true
autoincrement: true
- sourceColumn:
name: name
destinationColumn:
name: fname
options:
type_cast: varchar
length: 32
- sourceColumn:
name: surname
destinationColumn:
name: lname
options:
type_cast: varchar
length: 32
index: true
- sourceColumn:
name: age
destinationColumn:
name: age
options:
type_cast: int
- sourceColumn:
name: createdAT
destinationColumn:
name: created_at
options:
type_cast: datetime
- sourceColumn:
name: updatedAT
destinationColumn:
name: updated_at
options:
type_cast: datetime
Configs section
SourceConfigset the source database configurationsdbURIsource database URI
DestinationConfigset the destination database configurationsdbURIdestination database URI
Configs:
- SourceConfig:
dbURI: "postgres://root:[email protected]/oldDB" # set source database uri
- DestinationConfig:
dbURI: "mysql://root:[email protected]/newDB" # set destination database urimigrationTables section
migrationTablesin this configuration, you will write the source of the table that you have to migrate and the destination tables that will migrate the data.migrationTablespecify the source and destination table nameSourceTableinformation about source tablenamesource table name
DestinationTableinformation about destination tablenamedestination table namecreatebool value. This parameter tells the program whether it should create a table or not. (default false)
migrationTables:
- migrationTable:
SourceTable:
name: users
DestinationTable:
name: persons
create: TrueMigrationColumns section
MigrationColumnsspecify source and destination columnsourceColumninformation about source columnnamesource column name
destinationColumninformation about destination columnnamedestination column nameoptionscolumn optionstype_castdestination column type name varchar,integer etc. (when we convert data we use this parameter)
MigrationColumns:
- sourceColumn:
name: id
destinationColumn:
name: id
options:
type_cast: bigint
primary_key: true
autoincrement: trueIf you want to create a foreign key you can specify it in the column parameters
- sourceColumn:
name: USERID
destinationColumn:
name: user_id
options:
type_cast: uuid
foreign_key:
table_name: users
column_name: id
ondelete: CASCADEYou must create the main .yaml file and importing other files into main .yaml file.
main.yaml file
version: 1.1
Configs:
- SourceConfig:
dbURI: "mysql://admin:[email protected]/old"
- DestinationConfig:
dbURI: "postgresql://admin:[email protected]/new"
migrationTables:
- migrationTable: !import company.yaml
- migrationTable: !import op_cond.json
company.yaml file
SourceTable:
name: company
DestinationTable:
name: company
create: true
MigrationColumns:
- sourceColumn:
name: id
destinationColumn:
name: id
options:
primary_key: true
type_cast: uuid
- sourceColumn:
name: name
destinationColumn:
name: name
options:
length: 120
type_cast: varchar
nullable: false
- sourceColumn:
name: created
destinationColumn:
name: created
options:
type_cast: datetime
- sourceColumn:
name: updated
destinationColumn:
name: updated
options:
type_cast: datetime
nullable: trueop_conds.json file
{
"SourceTable": {
"name": "operation_conditions"
},
"DestinationTable": {
"name": "operation_conditions",
"create": true
},
"MigrationColumns": [
{
"sourceColumn": {
"name": "id"
},
"destinationColumn": {
"name": "id",
"options": {
"primary_key": true,
"type_cast": "uuid"
}
}
},
{
"sourceColumn": {
"name": "interest"
},
"destinationColumn": {
"name": "interest",
"options": {
"type_cast": "varchar",
"length": 30,
"nullable": false
}
}
},
{
"sourceColumn": {
"name": "FIFD"
},
"destinationColumn": {
"name": "FIFD",
"options": {
"type_cast": "varchar",
"length": 30,
"nullable": false
}
}
},
{
"sourceColumn": {
"name": "comission"
},
"destinationColumn": {
"name": "comission",
"options": {
"type_cast": "varchar",
"length": 30,
"nullable": false
}
}
}
]
}See also the list of contributors who participated in this project.
We are open to new ideas, additions. If you have any we would be happy to recieve and diccuss.
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent