Giter VIP home page Giter VIP logo

sqlite-jdbc's Introduction

SQLite JDBC Driver

GitHub Workflow Status (branch) Join the chat Maven Central javadoc Sonatype Nexus (Snapshots)

SQLite JDBC is a library for accessing and creating SQLite database files in Java.

Our SQLiteJDBC library requires no configuration since native libraries for major OSs, including Windows, macOS, Linux etc., are assembled into a single JAR (Java Archive) file.

Project Status

The project is maintained, but is not being actively developed:

  • We follow every new version of SQLite and will release a corresponding version of our driver.

  • Bugs will be investigated, and fixed if possible.

  • New features are driven by pull requests.

The current maintainer of the project is gotson.

Usage

➡️ More usage examples and configuration are available in USAGE.md

SQLite JDBC is a library for accessing SQLite databases through the JDBC API. For the general usage of JDBC, see JDBC Tutorial or Oracle JDBC Documentation.

  1. Download sqlite-jdbc-3.46.0.0.jar then append this jar file into your classpath.

  2. Download slf4j-api-1.7.36.jar then append this jar file into your classpath.

  3. Open a SQLite database connection from your code. (see the example below)

Example usage

Assuming sqlite-jdbc-3.46.0.0.jar and slf4j-api-1.7.36.jar are placed in the current directory.

> javac Sample.java
> java -classpath ".;sqlite-jdbc-3.46.0.0.jar;slf4j-api-1.7.36.jar" Sample   # in Windows
or
> java -classpath ".:sqlite-jdbc-3.46.0.0.jar:slf4j-api-1.7.36.jar" Sample   # in macOS or Linux
name = leo
id = 1
name = yui
id = 2

Sample.java

    import java.sql.Connection;
    import java.sql.DriverManager;
    import java.sql.ResultSet;
    import java.sql.SQLException;
    import java.sql.Statement;

    public class Sample
    {
      public static void main(String[] args)
      {
        // NOTE: Connection and Statement are AutoCloseable.
        //       Don't forget to close them both in order to avoid leaks.
        try
        (
          // create a database connection
          Connection connection = DriverManager.getConnection("jdbc:sqlite:sample.db");
          Statement statement = connection.createStatement();
        )
        {
          statement.setQueryTimeout(30);  // set timeout to 30 sec.

          statement.executeUpdate("drop table if exists person");
          statement.executeUpdate("create table person (id integer, name string)");
          statement.executeUpdate("insert into person values(1, 'leo')");
          statement.executeUpdate("insert into person values(2, 'yui')");
          ResultSet rs = statement.executeQuery("select * from person");
          while(rs.next())
          {
            // read the result set
            System.out.println("name = " + rs.getString("name"));
            System.out.println("id = " + rs.getInt("id"));
          }
        }
        catch(SQLException e)
        {
          // if the error message is "out of memory",
          // it probably means no database file is found
          e.printStackTrace(System.err);
        }
      }
    }

How does SQLiteJDBC work?

Our SQLite JDBC driver package (i.e., sqlite-jdbc-3.46.0.0.jar) contains three types of native SQLite libraries (sqlitejdbc.dll, sqlitejdbc.dylib, sqlitejdbc.so), each of them is compiled for Windows, macOS and Linux. An appropriate native library file is automatically extracted into your OS’s temporary folder, when your program loads org.sqlite.JDBC driver.

Supported Operating Systems

Since sqlite-jdbc-3.6.19, the natively compiled SQLite engines will be used for the following operating systems:

x86 x86_64 armv5 armv6 armv7 arm64 ppc64

Windows

macOS

Linux (libc)

Linux (musl)

Android

FreeBSD

In the other OSs not listed above, the pure-java SQLite is used. (Applies to versions before 3.7.15)

If you want to use the native library for your OS, build the source from scratch.

GraalVM native-image support

Sqlite JDBC supports GraalVM native-image out of the box starting from version 3.40.1.0. There has been rudimentary support for some versions before that, but this was not actively tested by the CI.

By default, the sqlitejdbc library for the compilation target will be included in the native image, accompanied by the required JNI configuration. At runtime, this library will be extracted to the temp folder and loaded from there. For faster startup however, it is recommended to set the org.sqlite.lib.exportPath property at build-time. This will export the sqlitejdbc library at build-time to the specified directory, and the library will not be included as a resource. As a result, the native image itself will be slightly smaller and the overhead of exporting the library at run-time is eliminated, but you need to make sure the library can be found at run-time. The best way to do this is to simply place the library next to the executable.

CLI example

native-image -Dorg.sqlite.lib.exportPath=~/outDir -H:Path=~/outDir -cp foo.jar org.example.Main

This will place both the sqlitejdbc shared library and the native-image output in the ~/outDir folder.

Maven example

This example uses the native-build-tools maven plugin:

<plugin>
    <groupId>org.graalvm.buildtools</groupId>
    <artifactId>native-maven-plugin</artifactId>
    <configuration>
        <buildArgs>
            <buildArg>-Dorg.sqlite.lib.exportPath=${project.build.directory}</buildArg>
        </buildArgs>
    </configuration>
</plugin>

This will automatically place the sqlitejdbc library in the /target folder of your project, creating a functional execution environment. When packaging the resulting app, simply include the library in the distribution bundle.

Download

Download from Maven Central or from the releases page.

<dependencies>
    <dependency>
      <groupId>org.xerial</groupId>
      <artifactId>sqlite-jdbc</artifactId>
      <version>3.46.0.0</version>
    </dependency>
</dependencies>

Snapshots of the development version are available in Sonatype’s snapshots repository.

Validating downloads

Maven Central resources are signed using GPG and the signature files, ending in .asc, are available in the same location as the other downloads.

The following key is currently used to sign releases:

-----BEGIN PGP PUBLIC KEY BLOCK-----
Comment: C1CB A75E C9BD 0BAF 8061  9354 59E0 5CE6 1818 7ED4
Comment: Taro L. Saito (For GitHub Actions) <[email protected]>

xjMEYuRVGhYJKwYBBAHaRw8BAQdA2Dp4m1Yhtb1g94pQzzL24FuP6b9KXF8lP9Dh
hZnynhfNM1Rhcm8gTC4gU2FpdG8gKEZvciBHaXRIdWIgQWN0aW9ucykgPGxlb0B4
ZXJpYWwub3JnPsKUBBMWCgA8FiEEwcunXsm9C6+AYZNUWeBc5hgYftQFAmLkVRoC
GwMFCwkIBwIDIgIBBhUKCQgLAgQWAgMBAh4HAheAAAoJEFngXOYYGH7UfPwBAK7x
TVRebZeWcAwmGaMUsbg7SgJou8xnkhByObPLUC/4AQDPsZeYmi4KXyXPzmqhCicd
Y+ZSJWIDQqitK2ujPDFXA844BGLkVRoSCisGAQQBl1UBBQEBB0Atu9kejBi+6wfO
T0a9z/LYEEdNXM/VX6xt1onKToPPdQMBCAfCeAQYFgoAIBYhBMHLp17JvQuvgGGT
VFngXOYYGH7UBQJi5FUaAhsMAAoJEFngXOYYGH7UlMABAKyRCazhVyUFg5FOpAnm
ckBY38CaMGPPLXVyY8Kr6dYFAP9wYLu7nsDZCOXkAgS+et4Pk1WZCggoYUkxsX1o
0KZXBQ==
=Wyas
-----END PGP PUBLIC KEY BLOCK-----

Project versioning explained

The project’s version follows the version of the SQLite library that is bundled in the jar, with an extra digit to denote the project’s increment.

For example, if the SQLite version is 3.39.2, the project version will be 3.39.2.x, where x starts at 0, and increments with every release that is not changing the SQLite version.

If the SQLite version is updated to 3.40.0, the project version will be updated to 3.40.0.0.

Hint for maven-shade-plugin

You may need to add shade plugin transformer to solve No suitable driver found for jdbc:sqlite: issue.

<transformer
	implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
	<resource>META-INF/services/java.sql.Driver</resource>
</transformer>
<dependency>
    <groupId>org.xerial</groupId>
    <artifactId>sqlite-jdbc</artifactId>
    <version>3.46.0.0</version>
</dependency>

How can I help?

We are always looking for:

  • Reviewers for issues or PRs, you can check GitHub labels

  • Contributors to submit PRs, you can check GitHub labels and GitHub labels

Please read our contribution guide.

sqlite-jdbc's People

Contributors

andyjefferson avatar b3nn0 avatar dependabot[bot] avatar doe300 avatar fatalmind avatar gbatumbya avatar gitblit avatar github-actions[bot] avatar gotson avatar jberkel avatar jcgay avatar joelcarranza avatar joshimoo avatar kares avatar kkriske avatar kroggen avatar michael-o avatar mkauf avatar mruddy avatar pyckle avatar reftel avatar snago avatar sualeh avatar tildedave avatar trohwer avatar witmoca avatar woehrl01 avatar xerial avatar yuvalp-k2view avatar zachee54 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

sqlite-jdbc's Issues

PreparedStatement setBytes()

Hello,

Somewhere between Release 3.8.10.2 and current code the PreparedStatement setBytes()
has been changed that results in what appears to be twice the byte size being stored in the
database. I'm not sure if getBytes() is the issue. This results in corrupt BLOB type being
stored.

A cursory look appears it maybe around the changes in the Kaufmann, (character > U+FFFF)
commit with the JDBC3PreparedStatement. I suppose it might also have to do with the native
library. Currently I'm using the Linux/64bit one. I have not tested on Windows or 32 bit machines.

danap.

Unsatisfied link CentOS linux

java.lang.UnsatisfiedLinkError: org.sqlite.core.NativeDB._open(Ljava/lang/String;I)V
at org.sqlite.core.NativeDB._open(Native Method) ~[xxx.jar:na]
at org.sqlite.core.DB.open(DB.java:161) ~[xxx.jar:na]
at org.sqlite.core.CoreConnection.open(CoreConnection.java:145) ~[xxx.jar:na]
at org.sqlite.core.CoreConnection.(CoreConnection.java:66) ~[xxx.jar:na]
at org.sqlite.jdbc3.JDBC3Connection.(JDBC3Connection.java:21) ~[xxx.jar:na]
at org.sqlite.jdbc4.JDBC4Connection.(JDBC4Connection.java:23) ~[xxx.jar:na]
at org.sqlite.SQLiteConnection.(SQLiteConnection.java:45) ~[xxx.jar:na]
at org.sqlite.JDBC.createConnection(JDBC.java:114) ~[xxx.jar:na]
at org.sqlite.JDBC.connect(JDBC.java:88) ~[xxx.jar:na]
at java.sql.DriverManager.getConnection(DriverManager.java:571) ~[na:1.7.0_85]
at java.sql.DriverManager.getConnection(DriverManager.java:187) ~[na:1.7.0_85]

gcc -v
Using built-in specs.
Target: x86_64-redhat-linux
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-bootstrap --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-languages=c,c++,objc,obj-c++,java,fortran,ada --enable-java-awt=gtk --disable-dssi --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-1.5.0.0/jre --enable-libgcj-multifile --enable-java-maintainer-mode --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --disable-libjava-multilib --with-ppl --with-cloog --with-tune=generic --with-arch_32=i686 --build=x86_64-redhat-linux
Thread model: posix
gcc version 4.4.7 20120313 (Red Hat 4.4.7-16) (GCC)

ldd --version
ldd (GNU libc) 2.12

java -version
java version "1.7.0_85"
OpenJDK Runtime Environment (rhel-2.6.1.3.el6_7-x86_64 u85-b01)
OpenJDK 64-Bit Server VM (build 24.85-b03, mixed mode)

uname -r
2.6.32-573.3.1.el6.x86_64

sqlite3 -version
3.6.20
Installed Packages
Name : sqlite
Arch : x86_64
Version : 3.6.20
Release : 1.el6_7.2

Improve perf by compiling with no locking and allowing users to configure it

We were investigating a concurrent performance issue with JRuby at jruby/jruby#3398 and after a few hours of investigation figured out that sqlite-jdbc is compiling sqlite with the default configuration of always threadsafe. Unfortunately it seems like that default mode does not treat separate database connections (to separate databases) as their own mutexes. As a result, parallel performance did not scale properly even when inserting into completely separate databases.

The fix, which is trivial, was to compile sqlite with SQLITE_THREADSAFE=0.

diff --git a/Makefile b/Makefile
index de1f1d3..31cd2d5 100644
--- a/Makefile
+++ b/Makefile
@@ -62,6 +62,7 @@ $(SQLITE_OUT)/sqlite3.o : $(SQLITE_UNPACKED)
            -DSQLITE_ENABLE_FTS3_PARENTHESIS \
            -DSQLITE_ENABLE_RTREE \
            -DSQLITE_ENABLE_STAT2 \
+           -DSQLITE_THREADSAFE=0 \
            $(SQLITE_FLAGS) \
            $(SQLITE_OUT)/sqlite3.c

Obviously this is not sufficient for the library. The full fix would be to build sqlite this way but when opening connections default to a thread-safe setup. As it turns out, sqlite can upgrade connections to be more threadsafe, but it can't downgrade them. The current configuration sets them by default to serialize all accesses.

The full doco for this is here: https://www.sqlite.org/threadsafe.html

Basically, sqlite-jdbc needs to configure the library to be thread-unsafe as a minimum, and then by default specify thread-safety when opening the connection. Users can then specify configuration parameters to open the connection as thread-unsafe (single thread access) giving them better concurrency if inserting into isolated databases.

I don't have a full patch for sqlite-jdbc, but I wanted to start the dialog now. We have a large number of JRuby users hitting sqlite via your library in heavily concurrent applications, and this would give them a better chance of having good scaling characteristics.

Redhat 6.3 sqlite-jdbc version

I'm testing a scala application on my machine (OX El Capitan 10.11.1) and running it in production on a redhat 6.3 x86_64 machine.

Redhat 6.3 ships with glibc 2.12, and AFAIK versions of sqlite-jdbc > 3.7.2 requires glibc 2.14. Using 3.7.2 I'm having several problems (no wal mode, no busy_timeout settings, concurrency issues etc...), and, anyway, it would be nice to use the latest version of the library :).

I think my only option is to recompile libsqlitejdbc.so for my redhat machine.
Is there a newer version of sqlite-jdbc that works fine with glibc 2.12?

I saw this problem was already mentioned here https://groups.google.com/forum/#!topic/xerial/XVwE2Ln7poo/discussion

SQLite driver support for CLOBs and BLOBs

Dear all, first of all thanks a lot for your efforts in the development of this driver.
I need to use it in a system in which it is mandatory to provide support for CLOBs and BLOBs data types.
I was wondering if you have any plans for supporting them and eventually what are the timescales.
Thanks a lot in advance,
Andrea

Question About DatabaseMetaData.getImportedKeys() and getPrimaryKeys()

Currently(v3.8.11.2), DatabaseMetaData.getImportedKeys() always return empty string on FK_NAME column, even if FKCOLUMN_NAME column is not empty. Is it better to return some non-empty string in FK_NAME column, especially if there exists more than one foreign keys.

Besides, DatabaseMetaData.getPrimaryKeys() always return null string on PK_NAME column, even if there exists primary key.

Foreign Keys with eclipselink

I'm trying to create a database with eclipselink and java persistence.
The framework creates some foreign keys for table joining but the SQLite will not accept it.
I guess SQLite doesn't support the ALTER TABLE ADD CONSTRAINT instruction.

This is the error:
[EL Warning]: 2015-08-17 15:22:24.052--ServerSession(1249457275)--Exception [EclipseLink-4002](Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: java.sql.SQLException: [SQLITE_ERROR] SQL error or missing database (near "CONSTRAINT": syntax error)
Error Code: 1
Call: ALTER TABLE MWE ADD CONSTRAINT FK_MWE_FORMERELEMENT_ID FOREIGN KEY (FORMERELEMENT_ID) REFERENCES MWE (ID)
Query: DataModifyQuery(sql="ALTER TABLE MWE ADD CONSTRAINT FK_MWE_FORMERELEMENT_ID FOREIGN KEY (FORMERELEMENT_ID) REFERENCES MWE (ID)")

Support to fill on Date with javabeans

It would be possible if you can transform internally the Long value of date to Date?
I did use Apache Commons DbUtils, and it has one feature that is awesome, the ability to create the objects with a BeanProcessor.

This can obtain the data from resultset and fill automatically using javabeans. It detects the data type to fill. And it work fine with all data and other Sql Drivers as MySQL, MariaDB, SqlServer.

But when I use with this library, I obtain an error when transform the Date. The problem is because can't transform the Long value of date to Date. So I need to do manually, but with a lot of fields, this is horrible =S

And sometimes occur problem with the Boolean value using this same technique. I'm not totally sure why this happens.

No such function: REGEXP

The SQLite grammar specification defines that there is support for usage of Regular expressions in any expr statement:

expr

The driver does not support this function:

A sample query:

SELECT * FROM SAMPLE WHERE row REGEXP 'ab?c*';

This is a sample stacktrace, when preparing a statement:

[SQLITE_ERROR] SQL error or missing database (no such function: REGEXP)
java.sql.SQLException: [SQLITE_ERROR] SQL error or missing database (no such function: REGEXP)
    at org.sqlite.core.DB.newSQLException(DB.java:890)
    at org.sqlite.core.DB.newSQLException(DB.java:901)
    at org.sqlite.core.DB.throwex(DB.java:868)
    at org.sqlite.core.NativeDB.prepare(Native Method)
    at org.sqlite.core.DB.prepare(DB.java:211)
    at org.sqlite.core.CorePreparedStatement.<init>(CorePreparedStatement.java:40)
    at org.sqlite.jdbc3.JDBC3PreparedStatement.<init>(JDBC3PreparedStatement.java:29)
    at org.sqlite.jdbc4.JDBC4PreparedStatement.<init>(JDBC4PreparedStatement.java:18)
    at org.sqlite.jdbc4.JDBC4Connection.prepareStatement(JDBC4Connection.java:47)
    at org.sqlite.jdbc3.JDBC3Connection.prepareStatement(JDBC3Connection.java:254)
    at org.sqlite.jdbc3.JDBC3Connection.prepareStatement(JDBC3Connection.java:226)

The stacktrace is cut, all classes below the snippet are my classes.

I am using org.xerial:sqlite-jdbc:3.8.11.2 straight from Maven Central.

Using of prepared batches leads to NPE

I'm getting NPE when using addBatch because internal batch variable at JDBC3PreparedStatement is still null at this point.

    public void addBatch() throws SQLException {
        checkOpen();
        batchPos += paramCount;
        if (batchPos + paramCount > batch.length) {
            Object[] nb = new Object[batch.length * 2];
            System.arraycopy(batch, 0, nb, 0, batch.length);
            batch = nb;
        }
        System.arraycopy(batch, batchPos - paramCount, batch, batchPos, paramCount);
    }

I snooped around JDBC3Statement and there the non-prepared version has this case already covered.

 public void addBatch(String sql) throws SQLException {
        internalClose();
        if (batch == null || batchPos + 1 >= batch.length) {
            Object[] nb = new Object[Math.max(10, batchPos * 2)];
            if (batch != null)
                System.arraycopy(batch, 0, nb, 0, batch.length);
            batch = nb;
        }
        batch[batchPos++] = sql;
    }

Not work in Raspbian(Raspberry Pi 2)

Hi,

It did not work with the latest version.
Linux ARM supported?

System info:
Linux raspberrypi 3.18.12-v7+ #782 SMP PREEMPT Tue Apr 28 19:54:13 BST 2015 armv7l GNU/Linux
jdk-8-oracle-arm-vfp-hflt

Error log:
[3.8.2]
java.lang.Exception: No native library is found for os.name=Linux and os.arch=arm
at org.sqlite.SQLiteJDBCLoader.loadSQLiteNativeLibrary(SQLiteJDBCLoader.java:284)
at org.sqlite.SQLiteJDBCLoader.initialize(SQLiteJDBCLoader.java:65)
at org.sqlite.core.NativeDB.load(NativeDB.java:53)
at org.sqlite.core.CoreConnection.open(CoreConnection.java:136)
at org.sqlite.core.CoreConnection.(CoreConnection.java:66)
at org.sqlite.jdbc3.JDBC3Connection.(JDBC3Connection.java:21)
at org.sqlite.jdbc4.JDBC4Connection.(JDBC4Connection.java:23)
at org.sqlite.SQLiteConnection.(SQLiteConnection.java:45)
at org.sqlite.JDBC.createConnection(JDBC.java:114)
at org.sqlite.JDBC.connect(JDBC.java:88)
at org.apache.commons.dbcp.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:38)
at org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:582)
at org.apache.commons.dbcp.BasicDataSource.validateConnectionFactory(BasicDataSource.java:1556)
at org.apache.commons.dbcp.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:1545)
at org.apache.commons.dbcp.BasicDataSource.createDataSource(BasicDataSource.java:1388)
at org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:1044)

[3.8.9]
java.lang.UnsatisfiedLinkError: org.sqlite.core.NativeDB._open(Ljava/lang/String;I)V
at org.sqlite.core.NativeDB._open(Native Method)
at org.sqlite.core.DB.open(DB.java:161)
at org.sqlite.core.CoreConnection.open(CoreConnection.java:218)
at org.sqlite.core.CoreConnection.(CoreConnection.java:74)
at org.sqlite.jdbc3.JDBC3Connection.(JDBC3Connection.java:24)
at org.sqlite.jdbc4.JDBC4Connection.(JDBC4Connection.java:23)
at org.sqlite.SQLiteConnection.(SQLiteConnection.java:45)
at org.sqlite.JDBC.createConnection(JDBC.java:114)
at org.sqlite.JDBC.connect(JDBC.java:88)
at org.apache.commons.dbcp.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:38)
at org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:582)
at org.apache.commons.dbcp.BasicDataSource.validateConnectionFactory(BasicDataSource.java:1556)
at org.apache.commons.dbcp.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:1545)
at org.apache.commons.dbcp.BasicDataSource.createDataSource(BasicDataSource.java:1388)
at org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:1044)

Default values are preferred over pragmas stated in filename

SQLiteConfig class sets default values such as dateStringFormat already in place when toProperties is called. This happens prematurely because extractPragmasFromFilename of CoreConnection thinks that these values are user defined because they are already stated in the given properties.

As a fix I propose hiding of the still public SQLiteConfig properties and moving the default login to getters where default value would be returned only when property hasn't yet been declared.

org/sqlite/SQLiteConfig.java b/src/org/sqlite/SQLiteConfig.java

@@ -43,16 +43,16 @@
 public class SQLiteConfig
 {
     private final Properties pragmaTable;
-    private int openModeFlag = 0x00;
+    private Integer openModeFlag;
     private TransactionMode transactionMode;
-    public final int busyTimeout;
+    private Integer busyTimeout;

     /* Date storage class*/
     public final static String DEFAULT_DATE_STRING_FORMAT = "yyyy-MM-dd HH:mm:ss.SSS";
-    public DateClass dateClass;
-    public DatePrecision datePrecision;
-    public long dateMultiplier;
-    public String dateStringFormat;
+    private DateClass dateClass;
+    private DatePrecision datePrecision;
+    private Long dateMultiplier;
+    private String dateStringFormat;

     /**
      * Default constructor.
@@ -68,28 +68,6 @@
      */
     public SQLiteConfig(Properties prop) {
         this.pragmaTable = prop;
-
-        String openMode = pragmaTable.getProperty(Pragma.OPEN_MODE.pragmaName);
-        if (openMode != null) {
-            openModeFlag = Integer.parseInt(openMode);
-        }
-        else {
-            // set the default open mode of SQLite3
-            setOpenMode(SQLiteOpenMode.READWRITE);
-            setOpenMode(SQLiteOpenMode.CREATE);
-        }
-        openMode = pragmaTable.getProperty(Pragma.SHARED_CACHE.pragmaName);
-        setOpenMode(SQLiteOpenMode.OPEN_URI); // Enable URI filenames
-
-        transactionMode = TransactionMode.getMode(
-                pragmaTable.getProperty(Pragma.TRANSACTION_MODE.pragmaName, TransactionMode.DEFFERED.name()));
-
-        dateClass = DateClass.getDateClass(pragmaTable.getProperty(Pragma.DATE_CLASS.pragmaName, DateClass.INTEGER.name()));
-        datePrecision = DatePrecision.getPrecision(pragmaTable.getProperty(Pragma.DATE_PRECISION.pragmaName, DatePrecision.MILLISECONDS.name()));
-        dateMultiplier = (datePrecision == DatePrecision.MILLISECONDS) ? 1L : 1000L;
-        dateStringFormat = pragmaTable.getProperty(Pragma.DATE_STRING_FORMAT.pragmaName, DEFAULT_DATE_STRING_FORMAT);
-
-        busyTimeout = Integer.parseInt(pragmaTable.getProperty(Pragma.BUSY_TIMEOUT.pragmaName, "3000"));
     }

     /**
@@ -190,7 +168,62 @@
      * @return The open mode flags.
      */
     public int getOpenModeFlags() {
+        Integer openModeFlag = this.openModeFlag;
+        if (openModeFlag == null) {
+            String openMode = pragmaTable.getProperty(Pragma.OPEN_MODE.pragmaName);
+            if (openMode != null) {
+                openModeFlag = Integer.parseInt(openMode);
+            }
+            else {
+                openModeFlag = 0x00;
+                // set the default open mode of SQLite3
+                openModeFlag |= SQLiteOpenMode.READWRITE.flag;
+                openModeFlag |= SQLiteOpenMode.CREATE.flag;
+            }
+            openMode = pragmaTable.getProperty(Pragma.SHARED_CACHE.pragmaName);
+            openModeFlag |= SQLiteOpenMode.OPEN_URI.flag; // Enable URI filenames
+        }
         return openModeFlag;
+    }
+    
+    public int getBusyTimeout() {
+        Integer busyTimeout = this.busyTimeout;
+        if (busyTimeout == null) {
+            busyTimeout = Integer.parseInt(pragmaTable.getProperty(Pragma.BUSY_TIMEOUT.pragmaName, "3000"));
+        }
+        return busyTimeout;
+    }
+    
+    public DateClass getDateClass() {
+        DateClass dateClass = this.dateClass;
+        if (dateClass == null) {
+            dateClass = DateClass.getDateClass(pragmaTable.getProperty(Pragma.DATE_CLASS.pragmaName, DateClass.INTEGER.name()));
+        }
+        return dateClass;
+    }
+    
+    public DatePrecision getDatePrecision() {
+        DatePrecision datePrecision = this.datePrecision;
+        if (datePrecision == null) {
+            datePrecision = DatePrecision.getPrecision(pragmaTable.getProperty(Pragma.DATE_PRECISION.pragmaName, DatePrecision.MILLISECONDS.name()));
+        }
+        return datePrecision;
+    }
+    
+    public long getDateMultiplier() {
+        Long dateMultiplier = this.dateMultiplier;
+        if (dateMultiplier == null) {
+            dateMultiplier = (datePrecision == DatePrecision.MILLISECONDS) ? 1L : 1000L;
+        }
+        return dateMultiplier;
+    }
+    
+    public String getDateStringFormat() {
+        String dateStringFormat = this.dateStringFormat;
+        if (dateStringFormat == null) {
+            dateStringFormat = pragmaTable.getProperty(Pragma.DATE_STRING_FORMAT.pragmaName, DEFAULT_DATE_STRING_FORMAT);
+        }
+        return dateStringFormat;
     }

     /**
@@ -208,11 +241,21 @@
      * @return The property object.
      */
     public Properties toProperties() {
-        pragmaTable.setProperty(Pragma.OPEN_MODE.pragmaName, Integer.toString(openModeFlag));
-        pragmaTable.setProperty(Pragma.TRANSACTION_MODE.pragmaName, transactionMode.getValue());
-        pragmaTable.setProperty(Pragma.DATE_CLASS.pragmaName, dateClass.getValue());
-        pragmaTable.setProperty(Pragma.DATE_PRECISION.pragmaName, datePrecision.getValue());
-        pragmaTable.setProperty(Pragma.DATE_STRING_FORMAT.pragmaName, dateStringFormat);
+        if (openModeFlag != null) {
+            pragmaTable.setProperty(Pragma.OPEN_MODE.pragmaName, Integer.toString(openModeFlag));
+        }
+        if (transactionMode != null) {
+            pragmaTable.setProperty(Pragma.TRANSACTION_MODE.pragmaName, transactionMode.getValue());
+        }
+        if (dateClass != null) {
+            pragmaTable.setProperty(Pragma.DATE_CLASS.pragmaName, dateClass.getValue());
+        }
+        if (datePrecision != null) {
+            pragmaTable.setProperty(Pragma.DATE_PRECISION.pragmaName, datePrecision.getValue());
+        }
+        if (dateStringFormat != null) {
+            pragmaTable.setProperty(Pragma.DATE_STRING_FORMAT.pragmaName, dateStringFormat);
+        }

         return pragmaTable;
     }
@@ -743,6 +786,11 @@
      * @return The transaction mode.
      */
     public TransactionMode getTransactionMode() {
+        TransactionMode transactionMode = this.transactionMode;
+        if (transactionMode == null) {
+            transactionMode = TransactionMode.getMode(
+                pragmaTable.getProperty(Pragma.TRANSACTION_MODE.pragmaName, TransactionMode.DEFFERED.name()));
+        }
         return transactionMode;
     }

org/sqlite/core/CoreConnection.java

@@ -65,15 +65,15 @@
         this.fileName = extractPragmasFromFilename(fileName, prop);

         SQLiteConfig config = new SQLiteConfig(prop);
-        this.dateClass = config.dateClass;
-        this.dateMultiplier = config.dateMultiplier;
-        this.dateFormat = FastDateFormat.getInstance(config.dateStringFormat);
-        this.dateStringFormat = config.dateStringFormat;
-        this.datePrecision = config.datePrecision;
+        this.dateClass = config.getDateClass();
+        this.dateMultiplier = config.getDateMultiplier();
+        this.dateFormat = FastDateFormat.getInstance(config.getDateStringFormat());
+        this.dateStringFormat = config.getDateStringFormat();
+        this.datePrecision = config.getDatePrecision();
         this.transactionMode = config.getTransactionMode();
         this.openModeFlags = config.getOpenModeFlags();

-        open(openModeFlags, config.busyTimeout);
+        open(openModeFlags, config.getBusyTimeout());

         if (fileName.startsWith("file:") && !fileName.contains("cache="))
         {   // URI cache overrides flags

Opening and closing of databases

Hi,

I think the implementation of Java_org_sqlite_core_NativeDB__1open() and Java_org_sqlite_core_NativeDB__1close()in NativeDB.c is wrong against the behaviour of SQLite databases:

  • In open() the db is attempted to be closed when it was already opened. This may lead to odd behaviour (despite the fact that it's a user error to attempt to open the db a second time). First of all I think there should just be an error thrown, but the db should not be be closed.
    Secondly, trying to close a db may or may not be successfull: If there is an unfinalized prepared statement the db will never be closed by SQLite. The result of sqlit3_close() must be checked in any case.
    If this is however for whatever reason intended behaviour that the db is closed, there should rather be sqlite3_close_v2() called, so that the db is closed when the last statement was finalized.
  • Similarly in close() the stored db handle must not be unset, and a meaningful error should be thrown (possible SQLite complains are again about not being able to close because of an unfinalized prepared statement, or else). So better leave the db open and keep the handle, until the db was really closed.

So I'd simple remove the sqlite3_close from within the open() method, and change close to this:

JNIEXPORT void JNICALL Java_org_sqlite_core_NativeDB__1close(
JNIEnv *env, jobject this)
{
int err = sqlite3_close(gethandle(env, this));
if (err == SQLITE_OK) {
//clearSpatialiteCacheForHandle(gethandle(env, this));
sethandle(env, this, 0);
}
else {
throw_errorcode(env, this, err);
}
}

Kind regards
Ben

Readups:
https://www.sqlite.org/c3ref/close.html

Use SQLSTATE codes

It would be helpful if sqlite-jdbc could map SQLite error codes and conditions to SQLSTATE codes so that it is possible to determine the reason for a SQLException without parsing the error message. A common use case for this are serialization and locking errors.

JDBC fail to get ARM native library

Hi, i come with a new problem with ARM native library call. It seems coming from this call
I use it with xowa, see this issue for the stack: gnosygnu/xowa#27

Error: java.lang.UnsatisfiedLinkError: org.sqlite.core.NativeDB._open(Ljava/lang/String;I)V
Stack: org.sqlite.core.NativeDB._open(Native Method)
org.sqlite.core.DB.open(DB.java:161)
org.sqlite.core.CoreConnection.open(CoreConnection.java:220)
org.sqlite.core.CoreConnection.<init>(CoreConnection.java:76)
org.sqlite.jdbc3.JDBC3Connection.<init>(JDBC3Connection.java:24)
org.sqlite.jdbc4.JDBC4Connection.<init>(JDBC4Connection.java:23)
org.sqlite.SQLiteConnection.<init>(SQLiteConnection.java:45)
org.sqlite.JDBC.createConnection(JDBC.java:114)
org.sqlite.JDBC.connect(JDBC.java:88)
java.sql.DriverManager.getConnection(DriverManager.java:664)
java.sql.DriverManager.getConnection(DriverManager.java:247)

this is the last post, If you have any advice... Thank you

date_string_format is treated differently when parsing and formatting

Due to the nature of how SimpleDateFormat works, the date_string_format parameter is treated differently when parsing and formatting.

Rail's ActiveRecord stores dates in the following format:
yyyy-MM-dd HH:mm:ss.SSS000
e.g.
2015-07-24 16:26:01.253000
NOTE: It's actually stored with millisecond precision, but 000 is always appended to the string to make it appear to be in microsecond precision.

If the above format string is passed to the sqlite-jdbc adapter, it will properly format strings, so that they can be written to the database, but it fails (and throws an exception) when it tries to read a date and parse it.

I have submitted a pull request which fixes this issue, but could use some help finalizing it:
#41

Support OSGI

I would like to use the library inside OSGi environment, and for it to work smoothly OSGi manifest should be added to the output JAR

Linux x86 Native Library

The current native library for Linux x86 is giving the following exception:

Exception in thread "AWT-EventQueue-0" java.lang.UnsatisfiedLinkError: org.sqlite.core.NativeDB._open(Ljava/lang/String;I)V
at org.sqlite.core.NativeDB._open(Native Method)
at org.sqlite.core.DB.open(DB.java:161)
at org.sqlite.core.CoreConnection.open(CoreConnection.java:224)
at org.sqlite.core.CoreConnection.(CoreConnection.java:78)
at org.sqlite.jdbc3.JDBC3Connection.(JDBC3Connection.java:24)
at org.sqlite.jdbc4.JDBC4Connection.(JDBC4Connection.java:23)
at org.sqlite.SQLiteConnection.(SQLiteConnection.java:45)
at org.sqlite.JDBC.createConnection(JDBC.java:114)
at org.sqlite.JDBC.connect(JDBC.java:88)
at java.sql.DriverManager.getConnection(DriverManager.java:571)
at java.sql.DriverManager.getConnection(DriverManager.java:187)
at com.dandymadeproductions.myjsqlview.gui.LoginFrame.accessCheck(LoginFrame.java:1191)

By reverting by to and earlier library, libsqlitejdbc.so size 858.1kb, from the 3.8.8.2 build
I believe fixes the error.

Dana M. Proctor
MyJSQLView Project Manager

Setting busy_timeout

I'm having issues using sqlite with multithreading. In particular I'm receiving a [SQLITE_BUSY] the database file is locked (database is locked) error.

I'm using sqlite-jdbc 3.8.11.2 on redhat 6.3.

I'm trying to set busy_timeout with no success. I'm using slick 3.1.0-RC2 for this.
I tried both using a simple query sqlu"pragma busy_timeout=10000" and from configuration like:

properties {
  busy_timeout=10000
}

In both approaches querying sqlite from the same thread and connection I get busy_timeout=0
I can successfuly set journal_mode = wal with this approach, I expected it to be the same with busy_timeout.

So I have 2 questions:

  1. Is busy_timeout=0 a reasonable default?
  2. Am I doing something wrong?

Native library doesn't load on ARM9 / causes app to abort

I'm having essentially this same issue as this resolved one --> #32

Except my processor is a Freescale i.MX28 / ARM926EJ-S running Debian OS Version 3.14.27 and JDK OpenJDK Runtime Environment (IcedTea 2.5.5) (7u79-2.5.5-1~deb7u1) / JamVM (build 2.0.0).

I hacked together a xerial sqlite-jdbc.jar using the native arm .so library discussed here ---> https://bitbucket.org/xerial/sqlite-jdbc/issue/79/compile-for-arm and that caused a stack trace saying the library in /tmp could not be opened.

When I use the most recent release (3.8.10.1) as-is, the app aborts with: 6414 Illegal instruction

Time Literal DateFormat Parsing

According to the jdbc4.0 ref. which I'm sure a new version is available, the following Date & Time
Literals have the format.

Date: {d, 'yyyy-mm-dd'}, Time:{t, 'hh:mm:ss'}, Timestamp: {ts, 'yyyy-mm-dd hh:mm:ss.f...'}

I would assume therefore if a TEXT data type field has the contents 2015-03-22 and contents are
returned via getDate() then I would not expect a date parse error would occur. Currently the
getDate() function is using a common DateFormat for all three of the time literals of
{yyyy-MM-dd HH:mm:ss.SSS}.

I would like to propose the SQLiteConfig class define the appropriate static instances of
DEFAULT_DATE_STRING_FORMAT, DEFAULT_TIME_STRING_FORMAT, & DEFAULT_TIMESTAMP_STRING_FORMAT along
with associated instances to be used in the resultSet returns for getDate(), getTime(), and
getTimeStamp(). It appears the setting methods are working appropriately.

If this is acceptable then I could perform the coding and submit for review and update. Perhaps
this issue is similar to open #18?

Thank you for your work in keeping the best option for the sqlite-jdbc current.

Dana Proctor
MyJSQLView Project Manager

Wrong UTF-8 decoding of Unicode Code Points higher then u+FFFF

When using UTF-8 encoding and insert a character where the unicode point is higher then u+FFFF. The Java ResultSet#getString() method return for this characters the same incorrect character. The Byte Representation seems to be correct.

I guess that that bug need to be in the C++ JNI Implementation https://github.com/xerial/sqlite-jdbc/blob/master/src/main/java/org/sqlite/core/NativeDB.c#L503

How to reproduce

  • Linux x64
  • org.xerial:sqlite-jdbc:3.8.11.2
  • java version 1.8.0_45 - Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)
# HINTS:
# - LANG=en_US.UTF-8

echo "CREATE TABLE TEST (id INTEGER PRIMARY KEY, name CHARSET);" | sqlite3 /tmp/sqlite.db -batch
echo "PRAGMA encoding = \"UTF-8\";" | sqlite3 /tmp/sqlite.db -batch

# LATIN CAPITAL LETTER A
# http://unicode-table.com/de/0041/
# http://www.fileformat.info/info/unicode/char/0041/index.htm
echo -e "INSERT INTO TEST (name) VALUES ('\x41');" | sqlite3 /tmp/sqlite.db -batch

# Miao Letter Archaic Ma
# http://unicode-table.com/de/16F06/
# http://www.fileformat.info/info/unicode/char/16F06/index.htm
echo -e "INSERT INTO TEST (name) VALUES ('\xF0\x96\xBC\x86');" | sqlite3 /tmp/sqlite.db -batch
Properties properties = new Properties();
properties.setProperty("characterEncoding", "UTF-8");
properties.setProperty("encoding", "\"UTF-8\"");
try (Connection connection = DriverManager.getConnection("jdbc:sqlite:/tmp/sqlite.db", properties)){
    Statement statement = connection.createStatement();
    ResultSet rs = statement.executeQuery("select name from TEST");
    while (rs.next()) {
        byte[] b = rs.getBytes("name");
        System.out.println("VALUE A = " + new String(b, java.nio.charset.StandardCharsets.UTF_8));

        String value = rs.getString("name");
        System.out.println("VALUE B = " + value);
    }
}

Hints

Maybe it's a CESU-8 (Modified UTF-8) vs UTF-8 encoding issue. According the JNI specification:

...Characters with code points above U+FFFF (so-called supplementary characters) are represented
 by separately encoding the two surrogate code units of their UTF-16 representation...

https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/types.html#modified_utf_8_strings
http://docs.oracle.com/javase/1.5.0/docs/guide/jni/spec/types.html#wp16542

Ideas

  • Probably the C++ JNI Code need to verify, if the database column value is in CESU-8 or UTF-8 encoded. And according that information it need to be converterd to a Java String (UTF-16) Object.
  • use C++ sqlite3_column_text16 or sqlite3_column_bytes16 function and NewString to create a Java String Object
  • Use PRAGMA encoding setting to decode VARCHAR Fields to Java String.

FYI

I reported that bug already on https://bitbucket.org/xerial/sqlite-jdbc/issues/200

UnitTests fail on Linux/Ubuntu 12.04 LTS 64 bits with UnsatisfiedLinkError

I've cloned the sqlite-jdbc and tried to build it, but got an UnsatisfiedLinkError running the Unit Tests.

Removing -fvisibility=hidden from Makefile.common fixed it.

System info:

  • Linux msdes076 3.2.0-70-generic #105-Ubuntu SMP Wed Sep 24 19:49:16 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
  • java version "1.6.0_33"
    OpenJDK Runtime Environment (IcedTea6 1.13.5) (6b33-1.13.5-1ubuntu0.12.04)
    OpenJDK 64-Bit Server VM (build 23.25-b01, mixed mode)
  • gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu5)
  • Apache Maven 3.2.3 (33f8c3e1027c3ddde99d3cdebad2656a31e8fdf4; 2014-08-11T22:58:10+02:00)

Raspberry pi load broken 3.8.11.1

Java error dump on initialization

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGILL (0x4) at pc=0xac6f8cb0, pid=5678, tid=3056833648
#
# JRE version: Java(TM) SE Runtime Environment (8.0-b132) (build 1.8.0-b132)
# Java VM: Java HotSpot(TM) Client VM (25.0-b70 mixed mode linux-arm )
# Problematic frame:
# C  [sqlite-3.8.11.1-7a260e63-0de6-4553-b0c4-4b87818e1de0-libsqlitejdbc.so+0x5cb0]  _init+0x290b
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.sun.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#

---------------  T H R E A D  ---------------

Current thread (0x00b5a7d8):  JavaThread "main" [_thread_in_native, id=5679, stack(0xb62ea000,0xb633a000)]

siginfo:si_signo=SIGILL: si_errno=0, si_code=1 (ILL_ILLOPC), si_addr=0xac6f8cb0

Registers:
  r0  = 0x00000008
  r1  = 0xbee4b814
  r2  = 0x0000015c
  r3  = 0x00000000
  r4  = 0x00000008
  r5  = 0xbee4b814
  r6  = 0x00d6ecb8
  r7  = 0x00bbaca8
  r8  = 0xbee4b814
  r9  = 0xb6f16048
  r10 = 0x00000008
  fp  = 0xb63372fc
  r12 = 0xb63374d0
  sp  = 0xb6337240
  lr  = 0xac6f63ab
  pc  = 0xac6f8cb0
  cpsr = 0xa0000030

Top of Stack: (sp=0xb6337240)
0xb6337240:   ac6f63a5 b6eff210 00bbaca8 00000001
0xb6337250:   00000000 00d6ecb8 00bbaca8 bee4b814
0xb6337260:   00000008 b6eff36c 00d6ecb8 b6f16560
0xb6337270:   00d6ee14 b6f16048 00000003 00000003
0xb6337280:   00000001 b6f033b8 00000001 b6f16094
0xb6337290:   00d6ecb8 00d6ecb8 80000000 00000000
0xb63372a0:   00000000 00000000 00071764 00000000
0xb63372b0:   00000000 000716a4 00000000 b6337298 

Instructions: (pc=0xac6f8cb0)
0xac6f8c90:   2018f8d8 65933b01 9c179b30 b021601c
0xac6f8ca0:   8ff0e8bd 0006626c 4a044b03 589b447b
0xac6f8cb0:   f7fdb10b 4770bca5 00075350 0000015c
0xac6f8cc0:   b5084a09 4b09447a 447b7812 4a08b95a 

Register to memory mapping:

  r0  = 0x00000008
0x00000008 is an unknown value

  r1  = 0xbee4b814
0xbee4b814 is an unknown value

  r2  = 0x0000015c
0x0000015c is an unknown value

  r3  = 0x00000000
0x00000000 is an unknown value

  r4  = 0x00000008
0x00000008 is an unknown value

  r5  = 0xbee4b814
0xbee4b814 is an unknown value

  r6  = 0x00d6ecb8
0x00d6ecb8 is an unknown value

  r7  = 0x00bbaca8
0x00bbaca8 is an unknown value

  r8  = 0xbee4b814
0xbee4b814 is an unknown value

  r9  = 0xb6f16048
0xb6f16048: _rtld_global+0 in /lib/ld-linux-armhf.so.3 at 0xb6ef0000

  r10 = 0x00000008
0x00000008 is an unknown value

  fp  = 0xb63372fc
0xb63372fc is pointing into the stack for thread: 0x00b5a7d8

  r12 = 0xb63374d0
0xb63374d0 is pointing into the stack for thread: 0x00b5a7d8

  sp  = 0xb6337240
0xb6337240 is pointing into the stack for thread: 0x00b5a7d8

  lr  = 0xac6f63ab
0xac6f63ab: <offset 0x33ab> in /tmp/sqlite-3.8.11.1-7a260e63-0de6-4553-b0c4-4b87818e1de0-libsqlitejdbc.so at 0xac6f3000

  pc  = 0xac6f8cb0
0xac6f8cb0: <offset 0x5cb0> in /tmp/sqlite-3.8.11.1-7a260e63-0de6-4553-b0c4-4b87818e1de0-libsqlitejdbc.so at 0xac6f3000



Stack: [0xb62ea000,0xb633a000],  sp=0xb6337240,  free space=308k
Native frames: (J=compiled Java code, j=interpreted, Vv=VM code, C=native code)
C  [sqlite-3.8.11.1-7a260e63-0de6-4553-b0c4-4b87818e1de0-libsqlitejdbc.so+0x5cb0]  _init+0x290b

Works fine in Windows 64bit

default support of json1 extension

Hi,

Are there any plans to integrate json1 extension to the library?
https://www.sqlite.org/json1.html

As I see, some libraries have already done that:
TryGhost/node-sqlite3#538

My case is the next, I use 0xDBE, installed the driver from the downloads page: https://bitbucket.org/xerial/sqlite-jdbc/downloads , however the json functions are not yet there.

Do we have any plans to integrate them? Or it is already somehow possible to use the json methods?

Regards,

Static/Dynamic linking of sqlite shadows symbols needed for RTree* callbacks

This issue effects mainly using deprecated functions in spatialite.

  • RTreeWithin
  • RTreeContains
  • RTreeIntersects
  • RTreeDistWithin

via the sqlite3_rtree_geometry_callback

If one does a search for RTreeIntersects they will find lots of

[SQLITE_ERROR] SQL error or missing database (no such function: RTreeIntersects)

And when traced through with a debugger the calls will seemingly disappear. Like in this issue (same effect, not xerial related)

What happens is that the JNI lib that xerial loads, calls to select load_extension('mod_spatialite') which then dynamically loads libsqlite makes a call to sqlite3_rtree_geometry_callback. The callbacks are registered in this second copy of sqlite, not the original. So when one attempts to call RTreeIntersects, the sqlite engine loaded by xerial knows nothing about it. Bermuda Triangle of function calls. The solution for those needing a quick hack is link in your libsqlite from the system.

In Makefile

-       $(CC) $(CFLAGS) -o $@ $(SQLITE_OUT)/*.o $(LINKFLAGS)
-       $(STRIP) $@
+       $(CC) $(CFLAGS) -o $@ $(SQLITE_OUT)/NativeDB.o $(LINKFLAGS)

in Makefile.common

-Default_LINKFLAGS := -shared
+Default_LINKFLAGS := -shared -L/usr/local/lib -lsqlite3

Migrating to GitHub

@gbatumbya

Just FYI. I migrated sqlite-jdbc's code base into GitHub from BitBucket. This is mainly because my preference. Recently there has been no commit activities in the sqlite-jdbc, so I simply wanted to move the repository close to where I am using frequently.

I am also going to migrate issue reports in the BitBucket.

Raspberry pi load broken 3.8.11.2

This picks up from issue #49 (Raspberry pi load broken 3.8.11.1). I see the same error metioned in #49 using sqlite-jdbc versions 3.8.11.1 and 3.8.11.2.

I assume that the Raspberry Pi is using the armhf/libsqlitejdbc.so. That library is the same in 3.8.11.1 and 3.8.11.2. Was it a simple deployment glitch?

I built my own sqlite-jdbc-3.8.11.3-SNAPSHOT.jar on my Raspberry Pi (Model B 2nd Revision) and it works. No changes to the source code. I simply ran a build outlined in the READ_BUILD.md file. The armhf/libsqlitejdbc.so generated is very different from 3.8.11.1 and 3.8.11.2. It also works.

$ cat /proc/cpuinfo
processor : 0
model name : ARMv6-compatible processor rev 7 (v6l)
BogoMIPS : 2.00
Features : half thumb fastmult vfp edsp java tls
CPU implementer : 0x41
CPU architecture: 7
CPU variant : 0x0
CPU part : 0xb76
CPU revision : 7

Hardware : BCM2708
Revision : 000e
Serial : 00000000bd950b8a
$ uname -a
Linux raspberrypi 4.1.7+ #817 PREEMPT Sat Sep 19 15:25:36 BST 2015 armv6l GNU/Linux

I'm happy to share my sqlite-jdbc-3.8.11.3-SNAPSHOT.jar if that would help.

Latest commit, test failure

Commit: 7121228 is giving me a test failure:

Running org.sqlite.PrepStmtTest
Tests run: 28, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.038 sec <<< FAILURE!

Prepared statements close/finalize on constraint violations.

Use case:
We are creating a prepared statement outside a loop. Inside the loop we bind the values to each parameter and then execute the prepared statement. The loop is running over data being read from an external source.

Issue:
If the prepared statement has an exception (e.g. constraint violation) the rest of the loop produces "statement is not executing" errors.

Cause:
This is caused by the statement being finalized in DB.java

execute(CoreStatement stmt, Object[] vals)

when step returns SQLITE_CONSTRAINT.

Workaround:
If an exception is thrown then recreate the prepared statement. This is not ideal as it means you can not easily use lambda's or try with resource.

Unsatisfied Link Error when REdeploying 3.8.7 in Tomcat 7.0.56

If I use 3.8.6 or 3.8.7 in a war file and undeploy and then deploy the application, I get an UnsatisfiedLinkError: org.sqlite.core.NativeDB._open(Ljava/lang/String;I)V. If I restart Tomcat everything works fine, so I guess it's actually not so bad for production, but for development it's frustrating.

This is with a Maven project, bundling the sqlite-jdbx jar inside the war. I haven't tried a separate jar (as was recommended for 3.8.6 with multiple applications). Should I try that?

This is on Centos 6.5 with OpenJDK 1.7.0_65.

Is this a known issue? A bug? Normal behaviour? Any help appreciated. Thanks.

Incompatibility setQueryTimeout() : JDK Statement and JDBC3Statement implementation

Hello,
I am facing a problem using JDBC3Statement sqlite implementation of Statement.

The JDK API says about Statement.setQueryTimeout()

          seconds - the new query timeout limit in seconds; zero means there is no limit.

The JDBC3Statement implementation do this :

    public void setQueryTimeout(int seconds) throws SQLException {
        if (seconds < 0)
            throw new SQLException("query timeout must be >= 0");
        conn.setBusyTimeout(1000 * seconds);
    }

By setting 0 to setQueryTimeout, I expected to have an unlimit busy timeout, but I got a totally diferent result.
Are you aware of this ?

Thanks,

Errorcode is not filled.

I have the following code:

import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.Statement;
import java.sql.SQLException;


public class CheckProverbsLocked {
    private CheckProverbsLocked() {
    }

    public static void main(String [] args) throws Exception {
        Connection   conn;
        Statement    stmt;

        Class.forName("org.sqlite.JDBC");
        conn = DriverManager.getConnection("jdbc:sqlite:proverbs.sqlite");
        stmt  = conn.createStatement();
        try {
            stmt.executeUpdate("begin immediate");
        } catch (SQLException e) {
            System.out.println(e.getErrorCode());
            System.out.println(e.getMessage());
            System.out.println(e.getSQLState());
        }
        stmt.close();
        conn.close();
    }

}

​I get the following output when the database is locked:
0
database is locked
null

​I would expect the first one to be 5​.

UnsatisfiedLinkError on ARM CORTEX A9 (soft float) at DB.open

I used sqlite-jdbc-3.8.11.jar to compile a jar that tries to open a connection to a database.
This is the error I get on a ARM CORTEX A9 (soft float).
The same jar is working fine on Ubuntu on x86. I think that working on ARM is quite important for a library like this.

zynq> java -jar sqlite-3.8.11.jar
Exception in thread "main" java.lang.UnsatisfiedLinkError: org.sqlite.core.NativeDB._open(Ljava/lang/String;I)V
at org.sqlite.core.NativeDB._open(Native Method)
at org.sqlite.core.DB.open(DB.java:161)
at org.sqlite.core.CoreConnection.open(CoreConnection.java:220)
at org.sqlite.core.CoreConnection.(CoreConnection.java:76)
at org.sqlite.jdbc3.JDBC3Connection.(JDBC3Connection.java:24)
at org.sqlite.jdbc4.JDBC4Connection.(JDBC4Connection.java:23)
at org.sqlite.SQLiteConnection.(SQLiteConnection.java:45)
at org.sqlite.JDBC.createConnection(JDBC.java:114)
at org.sqlite.JDBC.connect(JDBC.java:88)
at java.sql.DriverManager.getConnection(Unknown Source)
at java.sql.DriverManager.getConnection(Unknown Source)
at com.argonnetech.sqlite.SQLite.main(SQLite.java:14)

DatabaseMetaData.getColumns fails with NPE

If I create a table in SQLite (jdbc:sqlite:test1.db), say something like this
CREATE TABLE DOCUMENT
(
ID INTEGER PRIMARY KEY autoincrement,
"NAME" varchar(255) NULL
)

then I exit my app (and SQLite). I then restart the app and call
DatabaseMetaData.getColumns(null, null, null, null);

this gives
java.lang.NullPointerException
at org.sqlite.core.CoreDatabaseMetaData.escape(CoreDatabaseMetaData.java:182)
at org.sqlite.jdbc3.JDBC3DatabaseMetaData.getColumns(JDBC3DatabaseMetaData.java:1115)

I would expect it to return the columns for all tables present in this SQLite database.

Looking at the source code,
https://github.com/xerial/sqlite-jdbc/blob/master/src/main/java/org/sqlite/jdbc3/JDBC3DatabaseMetaData.java#L1115
it tries to escape a null as if there is a single table being returned from this method always? (or am I reading it wrong?)

This method has to be capable of returning many tables, and taking a "tableNamePattern" of null to mean don't restrict the tables (to be consistent with the JDBC spec, and all other JDBC drivers for other RDBMS). If I knew what the low level SQLite SQL would be to get such schema info I could try it in my own clone of SQLite-jdbc source. Any ideas/help?

sqlite dbcp

hello~:
I'm using dbcp by sqlite。

code:
BasicDataSource tempDbs = new BasicDataSource();
tempDbs.setDriverClassName("org.sqlite.JDBC");
tempDbs.setUrl("jdbc:sqlite:db/temp.db");
//tempDbs.setUsername("root");
//tempDbs.setPassword("root");
tempDbs.setInitialSize(20);
tempDbs.setMaxTotal(20);
tempDbs.setMaxIdle(50);
tempDbs.setMinIdle(5);
tempDbs.setMaxWaitMillis(1000);

            tempDbs.getConnection();

it catch a exception:
Exception in thread "main" java.sql.SQLException: Cannot create PoolableConnectionFactory (isValid() returned false)
at org.apache.commons.dbcp2.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:2291)
at org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:2038)
at org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:1533)
at com.dtdream.vehicle.util.ConnectionUtil.getTempConnection(ConnectionUtil.java:97)
at com.dtdream.vehicle.util.ConnectionUtil.main(ConnectionUtil.java:68)
Caused by: java.sql.SQLException: isValid() returned false
at org.apache.commons.dbcp2.PoolableConnection.validate(PoolableConnection.java:284)
at org.apache.commons.dbcp2.PoolableConnectionFactory.validateConnection(PoolableConnectionFactory.java:357)
at org.apache.commons.dbcp2.BasicDataSource.validateConnectionFactory(BasicDataSource.java:2304)
at org.apache.commons.dbcp2.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:2287)
... 4 more

jars : sqlite-jdbc-3.8.7.jar commons-dbcp2-2.1.jar commons-pool2-2.3.jar

pelase help!
thanks!

Library creates endless number of dll libraries in temp and they are not getting deleted

Hello there,

I noticed this day that my Windows system drive is out of space after running a java program I wrote which imported 500k different files into a database.

Inspecting the issue using Windows's "Disk Cleanup" shows around 129GB of data in user's temp folder, which turns out to be this sqlite jdbc dll driver, creating 500k different jdbc dlls on each call of the java my java program taking, each one taking 720kb of disk space.

The file name looks like this: sqlite-3.8.11.2-00a5dca3-f7cb-44b9-9014-de47cf377bd2-sqlitejdbc.dll where 00a5dca3-f7cb-44b9-9014-de47cf377bd2 part is always different.

After some analysis:
The DeleteOnExitHook is called but the library is not unloaded (I checked using ProcessExplorer) so it never gets deleted.

Couple of questions though. Why do you create a random UUID each time a process is created, it's obvious that you can have multiple instances of an application and you can have multiple reads from the same library as well since the library is loading in read-only mode. You can always have several sqlite native libraries as well since they are identified by the version number, so why the random UUID?

OS: Windows 7 64-bit
Java: 1.8.0_66 Java(TM) SE Runtime Environment (build 1.8.0_66-b18)
sqlite-jdbc: 3.8.11.2

A CoreConnection can not be shared between threads

CoreConnection as written is not thread-safe when parsing dates. This is due to the use of public final DateFormat dateFormat which uses SimpleDateFormat. SimpleDateFormat is known to be thread-unsafe.

This becomes a bigger issue with SQLitePooledConnection which generates a dynamic proxy of the CoreConnection. I think we should eliminate the DateFormat instance and generate one when it's needed from the configured datetime pattern.

I can put together a PR, if you wish.

createConnection shouldn't throw an exception if it isn't the appropriate driver.

According to the JDBC Driver spec:

Attempts to make a database connection to the given URL. The driver should return "null" if it realizes it is the wrong kind of driver to connect to the given URL. This will be common, as when the JDBC driver manager is asked to connect to a given URL it passes the URL to each loaded driver in turn. The driver should throw an SQLException if it is the right driver to connect to the given URL but has trouble connecting to the database.

However, the implementation is currently not returning null if it's the wrong kind of driver:

public static Connection createConnection(String url, Properties prop) throws SQLException {
  if (!isValidURL(url))
    throw new SQLException("invalid database address: " + url);
  url = url.trim();
  return new SQLiteConnection(url, extractAddress(url), prop);
}

If this driver is loaded along other drivers inside the JVM, we stack traces like this:

java.sql.SQLException: invalid database address: jdbc:postgresql://MYHOST/DB at org.sqlite.JDBC.createConnection(JDBC.java:111) at org.sqlite.JDBC.connect(JDBC.java:88) at java.sql.DriverManager.getConnection(DriverManager.java:571) at java.sql.DriverManager.getConnection(DriverManager.java:187)

JDBC4Connection not JDBC-compliant

The org.sqlite.jdbc4.JDBC4Connection returns null for all the unsupported features, like XML, CLOB, BLOB, ....
The documentation for java.sql.Connection specifies, that these methods should throw SQLFeatureNotSupportedException instead of returning null.

"Error opening connection" with sqlite-jdbc-3.8.7.jar (arm cortex a9 soft float)

I used sqlite-jdbc-3.8.7.jar to compile a jar that tries to open a connection to a database.
This is the error I get on a ARM CORTEX A9 (soft float). I also tried to use the sqlite-jdbc-3.8.11.jar but I get an UnsatisfiedLinkError exception and I will open another issue for this problem.

The same jars are working fine on Ubuntu on x86. I think that working on ARM is quite important for a this library.

zynq> java -jar sqlite-3.8.7.jar
java.sql.SQLException: Error opening connection
java.sql.SQLException: Error opening connection
at org.sqlite.core.CoreConnection.open(CoreConnection.java:140)
at org.sqlite.core.CoreConnection.(CoreConnection.java:66)
at org.sqlite.jdbc3.JDBC3Connection.(JDBC3Connection.java:21)
at org.sqlite.jdbc4.JDBC4Connection.(JDBC4Connection.java:23)
at org.sqlite.SQLiteConnection.(SQLiteConnection.java:45)
at org.sqlite.JDBC.createConnection(JDBC.java:114)
at org.sqlite.JDBC.connect(JDBC.java:88)
at java.sql.DriverManager.getConnection(Unknown Source)
at java.sql.DriverManager.getConnection(Unknown Source)
at com.argonnetech.sqlite.SQLite.main(SQLite.java:14)
Caused by: java.lang.Exception: No native library is found for os.name=Linux and os.arch=arm
at org.sqlite.SQLiteJDBCLoader.loadSQLiteNativeLibrary(SQLiteJDBCLoader.java:284)
at org.sqlite.SQLiteJDBCLoader.initialize(SQLiteJDBCLoader.java:65)
at org.sqlite.core.NativeDB.load(NativeDB.java:53)
at org.sqlite.core.CoreConnection.open(CoreConnection.java:136)
... 9 more

please release an update to bitbucket

Hi, could you put the new release:
News
2015 May 11th: sqlite-jdbc-3.8.10.1 release

to the bitbucket download section?
or do I have to compile it from source, because I don't use maven in my Project.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.