ishgroup / derbydump Goto Github PK

'INSERT INTO ... VALUES ...' is a good syntax, yet for large databases its possible that the generated statement will be even larger than 1Gb (mysql maximum possible length of query)

Safeguard is required, maybe by specifying another property
output.maxInsertSize = 16Mb

then inside the code the 'INSERT' statement has to be broken down into several smaller ones, each smaller than 16Mb.
(or throw exception if not possible, for example due to a large CLOB value)

Unless there is a severe error, there is no indication of any sort of what is happening (or what has happened) when running the app.

I see this in terminal regardless if the application succeeds or fails:

$ java -jar target/exporter-0.0.1-SNAPSHOT-jar-with-dependencies.jar
$

Did the application find the source database? Did it succeed to login? were there problems with file permissions to create output folder? no clue...

Just some sample output I'd like to see

$ java -jar target/exporter-0.0.1-SNAPSHOT-jar-with-dependencies.jar
DerbyDump starting, attempting to connect to database...
Cannot locate configuration file, please refer to README.MD
$

$ java -jar target/exporter-0.0.1-SNAPSHOT-jar-with-dependencies.jar
DerbyDump starting, attempting to connect to database...
Database connection unsuccessful, please verify the connection settings in

$

$ java -jar target/exporter-0.0.1-SNAPSHOT-jar-with-dependencies.jar
DerbyDump starting, attempting to connect to database...
Database connection successful, generating output...
Operation completed, output saved to
$

The specification states that the application is mean to to be a standalone program.

After 20 minutes of trying I cannot still figure out how to start up the program with a given set of parameters. (unless I edit 'src/main/resources/dump.properties' file and re-package the jar)

The dump.properties file should not be a part of distribution jar, but user provided file (preferably read from the directory where program is executed form)

Also some properties should have default values (ie. dump.buffer.size), and be only optional in the configuration/properties file.

The dump file has to be easily usable on any database, in particular a database with full schema, including foreign key indices.

Writing sql dump dealing with indexes might be hard, but luckily we only focus on mysql flavour, where such operations are easy.

I propose the following

• add another startup parameter called 'output.sql.flavour'. Default is mysql. We dont need any other flavour at the moment, but it will allow other opensource contributors to easily enhance this project.
• add another startup parameter called 'output.sql.ignoreFKConstraints'. Default to true.
• if ignoreFKConstraints == true and sql.flavour == mysql then adding
  'SET FOREIGN_KEY_CHECKS = 0;' on start and
  'SET FOREIGN_KEY_CHECKS = 1;' at the end of sql dump will do the trick.

I have java 6, gradlew is not compiling.

It should be at least mentioned in the readme that Java7 is needed.

Either your implementation of base64 encoding needs comprehensive testing, or (better) just use an existing library where this is properly implemented and tested.

http://commons.apache.org/proper/commons-codec//apidocs/org/apache/commons/codec/binary/Base64.html

This is really bad. StringBuilder must not be used by multiple threads:

http://docs.oracle.com/javase/1.5.0/docs/api/java/lang/StringBuilder.html

Java already has classes to do what you have reinvented from scratch such as... http://docs.oracle.com/javase/6/docs/api/java/io/FileWriter.html

If you used this, you would have no need to create multiple threads for reading and writing since Java will internally deal with the buffering for you.