Giter VIP home page Giter VIP logo

komodo's Introduction

The Teiid Designer project

Summary

This is the official Git repository for the Teiid Designer project.

Up to our recent 7.8 release this summer (2012) the codebase was housed in our SVN repository

Teiid Designer is an open source visual tool that enables rapid, model-driven definition, integration, management and testing of data services without programming using the Teiid runtime framework. With Teiid Designer, not only do you create source data models and map your sources to target formats using a visual tool, but you can also:

  • create a virtual database (or VDB) containing your models which you deploy to Teiid server and then access your data.
  • resolve semantic differences
  • create virtual data structures at a physical or logical level
  • use declarative interfaces to integrate, aggregate, and transform the data on its way from source to a target format which is compatible and optimized for consumption by your applications

This allows you to abstract the structure of the information you expose to and use in your applications from the underlying physical data structures. With Teiid Designer, data services are defined quickly, the resulting artifacts are easy to maintain and reuse, and all the valuable work and related metadata are saved for later reference.

You can use Teiid Designer to integrate multiple sources, and access them using the common data access standards:

  • Web Services / SOAP / XML
  • JDBC / SQL
  • ODBC / SQL

For more information on Teiid Desginer, including getting started guides, reference guides, and downloadable binaries, visit the project's website at http://www.jboss.org/teiiddesigner/ or follow us on our blog or on Twitter. Or hop into our IRC chat room and talk our community of contributors and users.

Get the code

The easiest way to get started with the code is to create your own fork of this repository, and then clone your fork:

$ git clone [email protected]:<you>/teiid-designer.git
$ cd teiid-designer
$ git remote add upstream git://github.com/Teiid-Designer/teiid-designer.git

At any time, you can pull changes from the upstream and merge them onto your master:

$ git checkout master               # switches to the 'master' branch
$ git pull upstream master          # fetches all 'upstream' changes and merges 'upstream/master' onto your 'master' branch
$ git push origin                   # pushes all the updates to your fork, which should be in-sync with 'upstream'

The general idea is to keep your 'master' branch in-sync with the 'upstream/master'.

For further information about developing in Eclipse and building using maven, please refer to this article.

Contribute fixes and features

Teiid Designer is open source, and we welcome anybody that wants to participate and contribute!

If you want to fix a bug or make any changes, please log an issue in the Teiid Designer JIRA describing the bug or new feature. Then we highly recommend making the changes on a topic branch named with the JIRA issue number. For example, this command creates a branch for the TEIIDDES-1234 issue:

$ git checkout -b teiddes-1234

After you're happy with your changes and a full build (with unit tests) runs successfully, commit your changes on your topic branch (using really good comments). Then it's time to check for and pull any recent changes that were made in the official repository:

$ git checkout master               # switches to the 'master' branch
$ git pull upstream master          # fetches all 'upstream' changes and merges 'upstream/master' onto your 'master' branch
$ git checkout mode-1234            # switches to your topic branch
$ git rebase master                 # reapplies your changes on top of the latest in master
                                      (i.e., the latest from master will be the new base for your changes)

If the pull grabbed a lot of changes, you should rerun your build to make sure your changes are still good. You can then either create patches (one file per commit, saved in ~/teiddes-1234) with

$ git format-patch -M -o ~/teiddes-1234 orgin/master

and upload them to the JIRA issue, or you can push your topic branch and its changes into your public fork repository

$ git push origin teiddes-1234         # pushes your topic branch into your public fork of ModeShape

and generate a pull-request for your changes.

We prefer pull-requests, because we can review the proposed changes, comment on them, discuss them with you, and likely merge the changes right into the official repository.

komodo's People

Contributors

blafond avatar elfilip avatar elvisisking avatar kylinsoong avatar mdrillin avatar mkralik3 avatar mmakovy avatar phantomjinx avatar tejones avatar vhalbert avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

komodo's Issues

Define Komodo "Library" concept

  1. locking and version control?
  2. define allowable content

Vdbs, Models, Views and Tables?
Data Sources
Teiid Servers
3) Define API for exporting/importing?
4) Define API to load/extract/copy to/from Workspace

WorkspaceManager RepositoryObserver Listener

In the constructor of WorkspaceManager, we are creating a RepositoryObserver. I think this listener gets notified of every repository event. A couple questions:

  • If the repository is not reachable, will events even be generated for this listener to handle?
  • Since WorkspaceManager is now just another KomodoObject, should we be caching them at all?

Seems like we might be able to get rid of this listener. Wdyt?

Provide means for each object type to expose a getAllowedChildTypes() method

The CLI's "create" action needs a way to determine what child object types are allowed for a given object.

VDB's, for instance, allow data roles, translator overrides, models and file entry object types.

Models allow tables, procedures, functions, views, etc.

So for instance, if a user is at a model node and types "create " they should see:
allowed types> table, view, storedProcedure, userDefinedFunction, pushdownFunction, virtualProcedure

Need to add property desrciptors for VDB elements that support specific <property> elements

The element in a vdb manifest supports a number of specific properties. Some defined as below:
name="query-timeout" value="256000"
name="allowed-languages" value="java, pascal"
name="security-domain" value="custom-security"
name="gss-pattern" value="%abc&a-b"
name="password-pattern" value="$xyz1-9"
name="authentication-type" value="USERPASSWORD"

Need to somehow create property descriptors for these specific properties so a UI can access them, present the "keys" to the user so they can set these specific properties.

The following elements support 0-man elements: vdb, model, translator and entry

Note that translator property descriptors may be available and accessed from the Teiid server. Only "overridden" or custom/user-defined properties need to be defined in the VDB.

Filter out non-teiid properties & nodes with jcr and mmcore prefixes

When using "property list" command I'm seeing jcr & mmcore prefixed properties. Need to filter the JCR-related properties in the CLI and maybe just remove the mmcore:, vdb:, ddl:, teiidddl: type prefixes

 jcr:mixinTypes           jcr:mixinTypes=[{http://www.modeshape.org/ddl/teiid/1.0}createTable]
 jcr:primaryType          jcr:primaryType={http://www.jcp.org/jcr/nt/1.0}unstructured
 jcr:uuid                 jcr:uuid=d593e7cc-d7a5-483d-93ca-721b3f528a16

 mmcore:modelType         mmcore:modelType=PHYSICAL

Export Should Quote OPTION Keys and Values

All OPTION keys should be exported by adding surrounding double quotes. This practice ensures case of identifier will be kept when round-trip importing. It also will make sure any embedded double quotes won't cause any issues. For instance, a key of Id should be exported as "Id" and a key of foo"bar should be exported as "foo""bar". Notice that embedded quotes must be escaped with an additional quote. We should also export foo.bar as "foo"."bar" as each individual segment is an identifier.

All OPTION values should be exported by adding surrounding single quotes. For instance, a value of myValue should be exported as 'myValue'. Quoting of values on export should always occur regardless if the value looks like a boolean, number, or string.

Exporting VDB with a model containing a Procedure without a result set ... results in javax.jcr.PathNotFoundException

A Procedure Result Set child is not always required. However the DdlNodeVisitor.procedure() method assumes it exists. This results in the exception below.

Caused by: javax.jcr.PathNotFoundException: The child "resultSet" could not be found under "/tko:komodo/tko:workspace/BooksVdb/BooksSource/getBookInfo" in workspace "komodoLocalWorkspace"
at org.modeshape.jcr.AbstractJcrNode.getNode(AbstractJcrNode.java:800)
at org.modeshape.jcr.AbstractJcrNode.getNode(AbstractJcrNode.java:1)
at org.komodo.modeshape.visitor.DdlNodeVisitor.procedure(DdlNodeVisitor.java:818)
at org.komodo.modeshape.visitor.DdlNodeVisitor.visit(DdlNodeVisitor.java:933)
... 44 more

Delete Legacy Classes

The org.komodo.relational.model.legacy packages/classes should be deleted from the org.komodo.relational and org.komodo.relational.test projects. They are no longer needed and should be deleted.

Add a feature that can import dynamic vdb.xml files into the workspace and maybe a library location

Migrating existing VDB artifacts, and dynamic vdbs in particular to a Komodo workspace will require an import capability. This would function similar to the ddl import feature, but load an xml file instead. The VDB, DDL and SQL sequencers should handle the full parsing of this VDB.xml file.

  • May want to expand this to handle VDB archives containing vdb.xml (manifest) and DDL model files.
  • Teiid Designer will handle converting EMF/XMI model VDB's to DDL vdbs.

Add KomodoObject validation framework

Thinking we may want this "plugable" or adaptable to KomodoObjects. Similar to developing relational objects with getters/setters and wrapping the JCR operations, we'd maybe need a Validatable interface that might have:

public interface Validatable {
KomodoStatus validate();
KomodoStatus getStatus();
}

Vdb could validate it's entire contents down to the "DDL" level, or have sub-validators for Permissions, Translators, etc....

Same goes for Models... Table, View (relational structure validators). Views would delegate to SQL validator etc.

Rewrite ObjectImpl.getChildrenOfType(UnitOfWork, String) To Use ObjectSearcher

Currently ObjectImpl queries for all children and then filters out the child nodes that don't match the specified type. Instead of obtaining all children first this method should use the ObjectSearcher to form a query that returns only the matching children from the repository.

I tried the following but it did not work:

final ObjectSearcher searcher = new ObjectSearcher(this.repository);
searcher.addFromType( type );
searcher.addWherePathClause( null, null, getAbsolutePath() );
final List< KomodoObject > result = searcher.searchObjects( transaction );

Implement Functionality In ValidationManager To Validate KomodoObjects

The ValidationManager API should be improved to handle the validation of one or more objects. The ValidationManager should collect, and then evaluate, all the applicable rules for each KomodoObject. The ability to cancel validation by using a progress monitor should be possible.

VdbExport does not handle all situations

The VdbExport functionality needs further testing using vdbs with the various child elements. It appears that the grouping nodes are currently not handled correctly.

Here is an example of an exception when I attempt to export a VDB in cli - using a vdb with DataRoles:

  • FAILED to export the MyVdb1 object.
    javax.jcr.PathNotFoundException: No item exists at path vdb:mappedRoleNames relative to /tko:komodo/tko:workspace/MyVdb1/vdb:dataRoles/MyDataRole1 in workspace "komodoLocalWorkspace"

Add KomodoObject.remove(UnitOfWork) Method

Currently there is only a removeChild method that takes one or more names as its arguments. Since JCR allows children of the same name, when calling removeChild it might be possible for you to delete the wrong node. This new method will allow a node to remove itself. JCR has a similar method on Node.

Add a DeleteCommand to CLI

Add a DeleteCommand and expose in the CLI

We probably want to present a "confirm" question for the user on the command line like "Confirm delete: my_vdb [vdb:virtualDatabase]" ?

Shell Should Display Paths Instead Of UUIDs

Currently when references are displayed in the shell, the UUID is displayed. We should display the object path instead as it provides more value to the user. One example where UUIDs are being shown is when the properties of a primary key are displayed. Primary keys have a multi-valued property of references to their columns.

CLI commands for creating PKs, ResultSets, etc need further investigation

Currently in the CLI, the following command can be issued to create a PK

  • create PrimaryKey PK1
    Need more thought into PKs and some other objects (like ResultSets) that are not really considered 'children' nodes. For example maybe they should be supported with a SET command instead of (or in addition to)

Also currently CREATE allows creation of multiple PKs. Need to improve the validation in CreateCommand to disallow creating more than one. Same goes for other similar objects

Can't set parameter direction to Direction.RETURN

Added unit test to ParameterImplTest to set parameter directon to Direction.RETURN. Received the following:

Note that Teiid doesn't support the RETURN value and has a VARIADIC value which Direction doesn't have.

org.komodo.spi.KException: javax.jcr.nodetype.ConstraintViolationException: The 'teiidddl:parameterType' property value 'RETURN' on node '/tko:komodo/tko:workspace/procedure/parameter' does not satisfy the constraints of the 'teiidddl:parameterType' property definition on the 'teiidddl:procedureParameter' node type, and no other property definition is valid.
at org.komodo.repository.ObjectImpl.handleError(ObjectImpl.java:715)
at org.komodo.repository.ObjectImpl.setProperty(ObjectImpl.java:1156)
at org.komodo.repository.ObjectImpl.setObjectProperty(ObjectImpl.java:1113)
at org.komodo.relational.internal.model.ParameterImpl.setDirection(ParameterImpl.java:373)
at org.komodo.relational.internal.model.ParameterImplTest.shouldHaveDirectionPropertyReturnValueAfterConstruction(ParameterImplTest.java:179)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
at org.junit.rules.RunRules.evaluate(RunRules.java:20)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
Caused by: javax.jcr.nodetype.ConstraintViolationException: The 'teiidddl:parameterType' property value 'RETURN' on node '/tko:komodo/tko:workspace/procedure/parameter' does not satisfy the constraints of the 'teiidddl:parameterType' property definition on the 'teiidddl:procedureParameter' node type, and no other property definition is valid.
at org.modeshape.jcr.AbstractJcrNode.setProperty(AbstractJcrNode.java:1780)
at org.modeshape.jcr.AbstractJcrNode.setProperty(AbstractJcrNode.java:1481)
at org.modeshape.jcr.AbstractJcrNode.setProperty(AbstractJcrNode.java:108)
at org.komodo.repository.ObjectImpl.setProperty(ObjectImpl.java:1073)
at org.komodo.repository.ObjectImpl.setProperty(ObjectImpl.java:1150)
... 32 more

Add A RemoveConstraintColumnCommand To The Shell

Currently there is a way to add a referenced column to a table constraint (see AddConstraintColumnCommand) but no way to remove them. It might be nice if the remove column command took either a UUID or a path (but at least a path).

DdlImporter interface improvements needed

The DDL importer automatically creates a new VDB with a new Model, then imports the DDL and puts the resulting objects into the model.

  • The interfaces should support passing in a Model, instead of automatically creating one.
  • for example in the cli I should be able to specify an existing model that the new DDL objects will go into.
  • [workspace]> import ./myDdlFile.txt /workspace/myvdb/myexistingmodel
  • After the importer improvements are finished, the ImportCommand can allow 'import ddl'. Currently the import ddl option is disabled.

Allow for specific relational properties (Cardinality, Precision, etc) that require a value greater than an int type can handle

Java int supports values of โ€“2,147,483,648 to 2,147,483,647.

Table cardinality can exceed 2,147,483,647. So we need to be able to handle a value much larger than 2,147,483,647. In this case changing to "long" java type would work.

long will take you up to 9,223,372,036,854,775,807 = 2^63 - 1. Once you need values over that, you can either look into BigInteger or just use a floating point number like double.

We just need to account for larger values and not assume "int" java type.

Expand create command to take raw DDL rather than simple relational arguments

Add capability to create an object using the following:

CREATE TABLE xyz (c1 integer, c2 string(255));

CREATE VIEW abc (c1 decimal, c3 long) AS SELECT * FROM table_b

Will require access to DDL Sequencer to parse/sequence general statements. Probably will need more direct user-feedback/validation for the provided statement/input.

Rewrite Shell Test Framework To Use The PlayCommand

Currently there is duplicate code in AbstractCommandTest and PlayCommand that reads and runs commands from a file. We should rewrite the test framework so that it constructs a PlayCommand to run the tests. One nice thing is that this change will constantly verify that the PlayCommand is working as it should be.

FindCommand Should Have An Object Name Argument

Currently the FindCommand has as its only argument the object type. When it executes it finds and returns all the object paths of the specified type. The result could be a huge number of paths depending on how big the workspace is. We should require a name argument, that accepts wildcards, to limit the search results. Also we might want to have a preference for max number of results returned.

Shell Recording Decision Should Be Moved To Shell

Right now, every command has to check after its execution to see if it should recording should be done. If we move this check to the shell (DefaultKomodoShell) the commands will not have to do this. This simplifies the command code and ensure all new commands will be recorded also.

Change CLI logging and labeling to show application name as VDBuilder

Since Komodo is not specific to anything relating to Teiid, let's change the labeling and logging info to show "VDBuilder" instead. This will help focus any discussions around usefulness for this first go-around.

We'll handle any change in package names/paths at a later date.

Create enum constants for possible Relational Object types and utility method for wrapping instanceof check to return integer type

Would be nice to have a universal/core way to identify KO types in any UI or editor. I've got a RelationalIdentifier class which does instanceof checks and returns an integer type value defined in my own UI constants file.

see: https://github.com/blafond/komodo/blob/master/plugins/org.komodo.eclipse.workspace/src/org/komodo/eclipse/workspace/CoreConstants.java
see: https://github.com/blafond/komodo/blob/master/plugins/org.komodo.eclipse.workspace/src/org/komodo/eclipse/workspace/util/RelationalIdentifier.java

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.