Giter VIP home page Giter VIP logo

aws-database-migration-samples's Introduction

aws-database-migration-samples

A set of sample database and associated items to allow customers to among other things follow along with published database migration recipes.

aws-database-migration-samples's People

Contributors

arunkumarthiags avatar jpeddicord avatar ramyakaushik avatar robins avatar ryanjmich avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

aws-database-migration-samples's Issues

Re-run PostgresSQL Script

For some reason, the installation script stops in the middle without exception. As a result, we find database objects are missing, such as generateticketactivity(), or null columns are not updated. This doesn't happen often, but it'd be great if we could have the re-install functionality.
Can you please modify the script to allow re-run ? We need to get the biggest table dms_sample.sporting_event_ticket dropped in the re-run as a higher priority, otherwise, it will keep growing and fill up the DB disk space quickly. Thanks.

Here are errors in the table drop at the moment:
psql:./schema/create-table.sql:35: ERROR: cannot drop table person because other objects depend on it
[ 14.989827] cloud-init[3218]: DETAIL: constraint set_person_id_997578592 on table sporting_event_ticket depends on table person
[ 14.990092] cloud-init[3218]: constraint tph_ticketholder_id_1077578877 on table ticket_purchase_hist depends on table person
[ 14.990355] cloud-init[3218]: constraint tph_transfer_from_id_1093578934 on table ticket_purchase_hist depends on table person
[ 14.990634] cloud-init[3218]: view sporting_event_ticket_info depends on table person
[ 14.990891] cloud-init[3218]: HINT: Use DROP ... CASCADE to drop the dependent objects too.
[ 15.001975] cloud-init[3218]: DROP TABLE
[ 15.017291] cloud-init[3218]: DROP TABLE
[ 15.022581] cloud-init[3218]: DROP TABLE
[ 15.023964] cloud-init[3218]: psql:./schema/create-table.sql:51: ERROR: cannot drop table sporting_event because other objects depend on it
[ 15.024147] cloud-init[3218]: DETAIL: view sporting_event_info depends on table sporting_event
[ 15.024351] cloud-init[3218]: view sporting_event_ticket_info depends on view sporting_event_info
[ 15.024631] cloud-init[3218]: HINT: Use DROP ... CASCADE to drop the dependent objects too.
[ 16.025992] cloud-init[3218]: psql:./schema/create-table.sql:55: ERROR: cannot drop table sporting_event_ticket because other objects depend on it
[ 16.026270] cloud-init[3218]: DETAIL: constraint tph_sport_event_tic_id_1061578820 on table ticket_purchase_hist depends on table sporting_event_ticket
[ 16.026403] cloud-init[3218]: view sporting_event_ticket_info depends on table sporting_event_ticket
[ 16.026546] cloud-init[3218]: HINT: Use DROP ... CASCADE to drop the dependent objects too.
[ 16.033511] cloud-init[3218]: DROP TABLE
[ 16.038725] cloud-init[3218]: DROP TABLE
[ 16.040107] cloud-init[3218]: psql:./schema/create-table.sql:67: ERROR: cannot drop table sport_location because other objects depend on it
[ 16.040258] cloud-init[3218]: DETAIL: constraint home_field_fk_565577053 on table sport_team depends on table sport_location
[ 16.040402] cloud-init[3218]: constraint se_location_id_fk_901578250 on table sporting_event depends on table sport_location
[ 16.040554] cloud-init[3218]: view sporting_event_info depends on table sport_location
[ 16.040697] cloud-init[3218]: view sporting_event_ticket_info depends on view sporting_event_info
[ 16.040864] cloud-init[3218]: HINT: Use DROP ... CASCADE to drop the dependent objects too.
[ 16.041383] cloud-init[3218]: psql:./schema/create-table.sql:71: ERROR: cannot drop table sport_team because other objects depend on it
[ 16.041649] cloud-init[3218]: DETAIL: constraint se_away_team_id_fk_885578193 on table sporting_event depends on table sport_team
[ 16.041780] cloud-init[3218]: constraint se_home_team_id_fk_869578136 on table sporting_event depends on table sport_team
[ 16.041959] cloud-init[3218]: view sporting_event_info depends on table sport_team
[ 16.042096] cloud-init[3218]: view sporting_event_ticket_info depends on view sporting_event_info
[ 16.042256] cloud-init[3218]: HINT: Use DROP ... CASCADE to drop the dependent objects too.
[ 16.042564] cloud-init[3218]: psql:./schema/create-table.sql:75: ERROR: cannot drop table sport_type because other objects depend on it
[ 16.042691] cloud-init[3218]: DETAIL: constraint st_sport_type_fk_581577110 on table sport_team depends on table sport_type
[ 16.042854] cloud-init[3218]: constraint se_sport_type_fk_853578079 on table sporting_event depends on table sport_type
[ 16.042988] cloud-init[3218]: HINT: Use DROP ... CASCADE to drop the dependent objects too.

Syntax error on mysql install script

I get the following error when I try to install DB using install-rds.sql:

ERROR 1064 (42000) at line 648 in file: './data/name_data.sql': You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'Insert into name_data (name_type,name) values ('MALE_FIRST','Agustin')' at line 2
Operation failed with exitcode 1

I am using MySQL workbench to run the script and my database is
db.m4.large RDS instance with MySQL engine version 5.7.22. Please look into this issue. Thanks in advance.

Bug in PostgresSQL generateticketactivity.sql

The PostgresSQL generateticketactivity.sql does not generate 1-6 tickets as documented in the readme. The line:
tick_quantity := floor(random()*(10000-2000+2000))+2000;

Should be:
tick_quantity := floor(random()*(6-1))+1;

mySQL install script errors

There are bunch of syntax and duplicate record errors that I came across while trying to install the mysql sample database.
I've tried to document most of them:

ERROR 1064 (42000) at line 1625 in file: './data/name_data.sql': You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'Insert into name_data (name_type,name) values ('LAST','Yoervis')' at line 2
Operation failed with exitcode 1

ERROR 1062 (23000) at line 1692 in file: './data/name_data.sql': Duplicate entry 'LAST-Mcdaniel' for key 'PRIMARY'
Operation failed with exitcode 1

ERROR 1062 (23000) at line 1721 in file: './data/name_data.sql': Duplicate entry 'LAST-Mckinney' for key 'PRIMARY'
Operation failed with exitcode 1

ERROR 1062 (23000) at line 1771 in file: './data/name_data.sql': Duplicate entry 'LAST-Mcgee' for key 'PRIMARY'
Operation failed with exitcode 1

ERROR 1062 (23000) at line 1861 in file: './data/name_data.sql': Duplicate entry 'LAST-Mcbride' for key 'PRIMARY'
Operation failed with exitcode 1

Oracle errors when running install-onprem.sql

I am logged into an installation of Oracle Database XE as user 'oracle' (as sysdba) on an EC2 instance. When I run install-onprem,sql all seems well except I get a few errors but I'm not sure if these are show stoppers?

ALTER DATABASE ADD SUPPLEMENTAL LOG DATA
Error at line 1:
ORA-31541: Supplemenmtal logging is not enabled in CDB$ROOT

ALTER DATABASE ADD SUPPLEMENTAL LOG DATA (PRIMARY LKEY) COLUMNS
Error at line 1:
ORA-31541: Supplemenmtal logging is not enabled in CDB$ROOT
- Is supplemental logging necessary?

grant select on sys.DBMS_LOGMNR to dms_user
Error at line 1:
ORA-02225: only EXECUTE and DEBUG priviligese are valid for procedures
- will this cause issues?

drop table ticket_purchase_hist
Error at line 1:
ORA-00942: table or view does not exist
- Is this just a cleanup incase this has already been installed or should this table be there and something went wrong further up?

PostgreSQL script does not create schema

The script aws-database-migration-samples/PostgreSQL/sampledb/v1/install-postgresql.sql does not create the dms_sample schema. It should either create it or ask the user to do so before running the script.

EMPTY TABLE IN DMS POSTGRES MIGRATION

we configured dms service source as ecs and destination as rds , Here all tables are migrated well except one , That exception table we are getting as empty table.I shared cloudwatch logs here can you please help.

image

MySql script not idempotent

Running aws-database-migration-samples/mysql/sampledb/v1/install-rds.sql failed due to [this bug]. Fixing it was simple enough, but re-running required a number of changes since the scripts are not idempotent. Ideally, there'd be a DROP TABLE IF EXISTS table_name; before each CREATE TABLE, and DROP USER IF EXISTS before the dms_user is created.

AWS DMS task stuck in between

Hi,
I am migrating data from Oracle11g DB (~1 GB) to AWS RDS PostgreSQL 9.5 using AWS DMS.
Using Replication instance - c4.xlarge, RDS - m4.xlarge.

No of Tables : 29
Contains BLOB data type for 3 tables
Task parameters

  • Target table preparation mode - Do nothing
  • Include LOB columns in replication - Limited LOB mode
  • Include LOB columns in replication - 5
  • Transaction consistency timeout - 600 second
  • commit rate during full load - 10000

When executed task for all 29 tables task gets stuck after 5 mins, without any updates in task-log or any progress in data migration (only 10 tables completed migration and 3 tables in queue)

However, when divided into 3 different tasks, data get migrated successfully.

Is any parameter configured incorrectly for task or DB.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.