Dependencies

• exasol/virtual-schema-common-lua#60

Acceptance Criteria

1. Push-down supports IS [NOT] JSON (see https://docs.exasol.com/db/latest/sql_references/predicates/is[not]json.htm)

Situation

The current user guide is from RLS Lua. While the installation instructions are similar, the rest of the user guide must come from the existing Exasol user guide.

Acceptance Criteria

1. Installation instructions ported from RLSL
2. Rest ported from Exasol VS Java

Situation

In #23 we realized that TLS certificate validation is not yet possible in Lua, because Lua has no file system access and thus cannot read from the certificate store.

Since this requires a database change, we decided to create this follow-up ticket of #23 to add certificate validation support once the database is prepared for this.

Acceptance Criteria

• An integration test proves that a TLS connection requires a valid peer certificate in order to establish a connection.
• Event-driven way to find out when to restart the database after installing a certificate (install_test_certificate.sh).

https://github.com/exasol/exasol-virtual-schema-lua/blob/main/doc/user_guide/user_guide.md

Typo in adapter schema name

CREATE SCHEMA EVSL_SCHEMA;
...
CREATE OR REPLACE LUA ADAPTER SCRIPT EVLS_SCHEMA.EVSL_ADAPTER AS
'''

Situation

Join and sub-select rendering are available in the base library, but the dialect does not report the capabilities and integration tests are missing.

Sub-selects are unfortunately not supported by the core database, so the adapter can only handle joins.

Acceptance Criteria

1. Integration tests prove joins are supported

Tasks

• Add system specification
• Add design

Situation

We split the namespace change off into a separate task to make reviewing easier.

Acceptance Criteria

• Namespace switched to exasol.evsl

Situation

Metadata access to the remote data source during CREATE SCHEMA requires a network connection. This connection should be protected by TLS.

The following task requires a change in the Exasol database because Lua does not yet have access to the certificate store.

• ❌ Run the same test with a valid certificate and get it green → Follow up ticket: #36

Acceptance Criteria

• TLS support available TLS support without certificate validation.

Situation

In the first MVP of the Exasol Virtual Schema we only supported reading metadata from a local Exasol database. The first step in enabling access to a remote Exasol database is to be able to read the metadata using the Lua Exasol Driver.

Acceptance Criteria

1. Users can define a remote connection
2. Exasol can read the metadata for a remote connection
3. An integration test proves that reading the remote metadata works
4. Properties are backward-compatible (ignoring is OK) with the existing Java-variant
5. Connection definitions are backward-compatible with the existing Java-variant

It is okay though if the test accesses the same database as long as it uses the database driver to do it. Since IMPORT is not in the scope of this ticket, this kind of test is the easiest way to decouple the tickets.

Implementation hints

Check "Accessing Connection Definitions" in the online handbook for details on how to read the contents of connection definitions in a Lua script.

Situation

We already have an Exasol Virtual Schema written in Java. The Lua variant must be interface compatible, so all existing integration tests must work with the Lua VS too

Tasks

• Copy the integration test
• Fix failing tests (i.e. implement the Virtual Schema)

Summary

When users combine integer constants as expressions with DISTINCT, the engine pushes down GROUP BY <integer-constant>. Exasol interprets this as "group by column number", which yields wrong results. What should happen here is that you get a result with a single group. GROUP BY TRUE, GROUP BY '0' or any other non-integer constant will work as expected.

Note that some databases interpret this as "group by integer constant". So the behavior is database specific. The reason is that this case is not covered by the SQL standard.

Workaround:

The adapter should work around this by turning GROUP BY <integer-constant> into GROUP BY <other-constant>.

Theoretically that should be done on a db-by-db basis, but that is too much effort, so a general workaround in VSCL and VSCJ should be implemented. This can be overridden in the dialect if there is ever the need.

Features

• Turns GROUP BY <integer-constant> into GROUP BY <other-constant>

• org.eclipse.parsson:parsson:jar:1.1.1 in test
  • CVE-2023-4043, severity CWE-20: Improper Input Validation (7.5)

Vulnerability is typically introduced via transitive test dependency exasol-test-containers:

\- com.exasol:exasol-testcontainers:jar:6.6.2:test
   \- com.exasol:bucketfs-java:jar:3.1.0:test
      \- org.eclipse.parsson:parsson:jar:1.1.1:test

Dependencies

• exasol/virtual-schema-common-lua#19
• exasol/virtual-schema-common-lua#67

Acceptance Criteria

1.Pushdown supports LISTAGG (see https://docs.exasol.com/db/latest/sql_references/predicates/is[not]json.htm)

Situation

In #21 we implemented import from a remote Exasol database. So far the IMPORT statement that the adapter generates does not contain a column type list. While that works for getting the data, it has the downside of letting the ExaLoader pick the column types. Those do not always match the ones the database expects.

What we need is to take the data types the core database in the request element selectListDataTypes and explicitly state them in the IMPORT.

Acceptance Criteria

• IMPORT contains wanted result column data types

Situation

Currently in file QueryRuntimeIT we only test the absolute maximum runtime. This is a good regression test.
But it does not fulfill the criteria defined in qr~query-execution-time~1.

Acceptance Criteria

1. Test exists that establishes the Query time baseline by querying the source schema and compares this with the same query through the Virtual Schema as defined in qr~query-execution-time~1.

Situation

The EVSL has a good number of integration tests already. Still we should scan the existing Java EVS and port any test cases that are not yet covered to ensure backward compatibility

Acceptance Criteria

• Integration tests for functions not covered yet ported from the Java variant.

Summary

Loop with Unreachable Exit Condition ('Infinite Loop') vulnerability in Apache Commons Compress.This issue affects Apache Commons Compress: from 1.3 through 1.25.0.

Users are recommended to upgrade to version 1.26.0 which fixes the issue.

CVE: CVE-2024-25710
CWE: CWE-835

References

• https://ossindex.sonatype.org/vulnerability/CVE-2024-25710?component-type=maven&component-name=org.apache.commons%2Fcommons-compress&utm_source=ossindex-client&utm_medium=integration&utm_content=1.8.1
• http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2024-25710
• https://lists.apache.org/thread/cz8qkcwphy4cx8gltn932ln51cbtq6kf

Situation

We need to support nanosecond timestamp resolution.

Acceptance Criteria

• An integration test against Exasol 8 proves that TS(9) is supported

Acceptance Criteria

For each of the listed predicates an integration test exists:

• FN_PRED_AND
• FN_PRED_BETWEEN
• FN_PRED_EQUAL
• FN_PRED_IN_CONSTLIST
• FN_PRED_IS_JSON
• FN_PRED_IS_NOT_JSON
• FN_PRED_IS_NOT_NULL
• FN_PRED_IS_NULL
• FN_PRED_LESS
• FN_PRED_LESSEQUAL
• FN_PRED_LIKE
• FN_PRED_LIKE_ESCAPE
• FN_PRED_NOT
• FN_PRED_NOTEQUAL
• FN_PRED_REGEXP_LIKE
• FN_PRED_OR

Situation

After we solved #20, we can then implement creating IMPORT statements for getting data from a remote Exasol.

Dependencies

• #20
• exasol/virtual-schema-common-lua#59

Acceptance Criteria

1. An integration tests proves that EVSL can import data from a remote Exasol database

Situation

Tests with a development version of V8 are only available locally. Currently, the CI tests run with 7.1.18 and skip remote tests.
Once V8 is on Dockerhub, we need to reenable CI tests with V8 in ExasolVritualSchemaTestConstants.

Acceptance Criteria

• CI tests run with Exasol version that supports remote login via Lua.

Summary

Allocation of Resources Without Limits or Throttling vulnerability in Apache Commons Compress.This issue affects Apache Commons Compress: from 1.21 before 1.26.

Users are recommended to upgrade to version 1.26, which fixes the issue.

CVE: CVE-2024-26308
CWE: CWE-770

References

• https://ossindex.sonatype.org/vulnerability/CVE-2024-26308?component-type=maven&component-name=org.apache.commons%2Fcommons-compress&utm_source=ossindex-client&utm_medium=integration&utm_content=1.8.1
• http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2024-26308
• https://lists.apache.org/thread/ch5yo2d21p7vlqrhll9b17otbyq4npfg
• https://www.openwall.com/lists/oss-security/2024/02/19/2

Situation

To unify the code base of EVSL and RLSL we need to move common parts to a library called exasol-virtual-schema-common-lua (EVSCL).

Situation

Exasol Virtual Schema Lua started as a port of RLS Lua. Simply put it is RLS minus the row-restriction logic. In this issue we will remove the RLS-specific parts of the ported software.

Acceptance Criteria

1. RLS logic removed
2. RLS mentions in documentation removed

Acceptance Criteria

1. An integration tests proves that filter expressions (capability FILTER_EXPRESSIONS) are supported.