Giter VIP home page Giter VIP logo

cinchapi / concourse Goto Github PK

View Code? Open in Web Editor NEW
315.0 66.0 234.0 52.18 MB

Distributed database warehouse for transactions, search and analytics across time.

Home Page: http://concoursedb.com

License: Apache License 2.0

Java 92.45% Python 1.02% Ruby 1.50% Shell 0.74% Thrift 1.55% CSS 0.04% HTML 0.21% Groovy 0.91% Makefile 0.01% PHP 1.29% Smarty 0.01% Roff 0.27% Dockerfile 0.01%
database nosql analytics search backend platform

concourse's Introduction

Concourse

Join the chat at https://gitter.im/cinchapi/concourse

Concourse is a distributed database warehouse for transactions search and analytics across time. Developers prefer Concourse because it simplifies building misssion-critical systems with on-demand data intelligence. Furthermore, Concourse makes end-to-end data management trivial by requiring no extra infrastructure, no prior configuration and no continuous tuning–all of which greatly reduce costs, and allow developers to focus on core business problems.

This is version 0.12.0 of Concourse.

Quickstart

Docker

Using Concourse via Docker is the quickest way to get started.

Run concourse

docker run -p 1717:1717 --name concourse cinchapi/concourse

NOTE: This will run Concourse in the foreground. To run in the background, add a -d flag to the docker run command.

Run concourse with a peristent/shared directory

docker run -p 1717:1717 -v </path/to/local/data>:/data --name concourse cinchapi/concourse

Run concourse with a custom HEAP_SIZE

docker run -p 1717:1717 -e CONCOURSE_HEAP_SIZE=<HEAP_SIZE> --name concourse cinchapi/concourse

Run concourse shell and connect to the running concourse docker container

docker run -it --rm --link concourse:concourse cinchapi/concourse shell --host concourse --password admin

Run concourse shell and connect to a running concourse container spun up using docker-compose

docker-compose run concourse shell --host concourse

Use the concourse import to perform an interactive import that reads input from the command line

docker run -it --rm --link concourse:concourse cinchapi/concourse import --host concourse --password admin

Use the concourse import to import a file from the host machine into the concourse docker container

xargs -I % docker run -i --rm --link concourse:concourse --mount type=bind,source=%,target=/data/% cinchapi/concourse import --host concourse --password admin -d /data/% <<< /absolute/path/to/file

Run server-side management commands (e.g. concourse debug) within the running container

docker exec -it concourse concourse <command> <args>

For example, you can call the concourse users sessions command

docker exec -it concourse concourse users sessions --password admin

For more information, visit https://docs.cinchapi.com/concourse/quickstart.

Usage

Let's assume we have the an array of JSON objects corresponding to NBA players. NOTE: These examples assume you're using Concourse Shell, but are easily adaptable to any of the Concourse client drivers or REST API.

data = '[
    {
        "name": "Lebron James",
        "age": 30,
        "team": "Cleveland Cavaliers"
    },
    {
        "name": "Kevin Durant",
        "age": 26,
        "team": "OKC Thunder"
    },
    {
        "name": "Kobe Bryant",
        "age": 36,
        "team": "LA Lakers"
    }
]'
You can use Concourse to quickly insert the data and do some quick analysis. Notice that we don't have to declare a schema, create any structure or configure any indexes.
ids = insert(data) // each object is added to a distinct record
lebron = ids[0]
durant = ids[1]
kobe = ids[2]
You can read and modify individual attributes without loading the entire record.
get(key="age", record=kobe)
add(key="name", value="KD", record=durant)
remove(key="jersey_number", value=23, record=lebron)
You can easily find records that match a criteria and select the desired since everything is automatically indexed.
select(criteria="team = Chicago Bulls")
select(keys=["name", "team"], criteria="age bw 22 29")
You can even query data from the past without doing any extra work.
// Return data from 04/2009 from records that match now
get(key="age", record=durant, time=time("04/2009")) 

// Return records that matched in 2011
find("team = Chicago Bulls at 2011") 

// Return data from two years ago from records that match now
select(criteria="age > 25 and team != Chicago Bulls", time=time("two years ago")) 
It is very easy to analyze how data has changed over time and revert to previous states.
// Analyze how data has changed over time and revert to previous states
audit(key="team", record=lebron)
revert(key="jersey_number", record=kobe, time=time("two years ago"))
And ACID transactions are available for important, cross record changes.
stage
set(key="current_team", value="OKC Thunder", record=lebron)
set(key="current_team", value="Cleveland Cavs", record=durant)
commit

...or using shorthand syntax

stage({
    set(key="current_team", value="OKC Thunder", record=lebron);
    set(key="current_team", value="Cleveland Cavs", record=durant);
})

You can find more examples in the examples directory. More information is also available in the Concourse Guide.

Motivation

Whether you use SQL or NoSQL, it's hard to get real-time insight into your mission critical data because most systems are only optimized for either transactions or analytics, not both. As a result, end-to-end data management requires complex data pipelining, which slows down development, complicates infrastructure and increases costs.

The Solution

Concourse is an integrated and self-managing transactional database that enables real time ad-hoc analytics without any configuration.

Automatic Indexing

You no longer need to plan queries in advance because Concourse automatically indexes all of your data while guaranteeing constant time writes that are super fast. Concourse fully supports ad-hoc range and predicate queries and automatically caches frequently requested data for optimal performance.

Version Control

Concourse automatically tracks changes to your data, just like Git does for source code. Of course this means that you can easily audit changes and revert to previous states without downtime; but it also means that you have the power to query data from the past. Version control in Concourse makes it possible to build applications that know what was known when and can analyze real-time changes over time.

ACID Transactions

Concourse supports truly distributed ACID transactions without restriction. And we use dynamic resource management with just-in-time locking to ensure that they deliver both the highest performance and strongest consistency. So no need to guess when your data will eventually become consistent. When distributed Concourse responds to a query, you can trust the results immediately.

Simple Data Model

Concourse's document-graph structure is lightweight and flexible–it supports any kind of data and very large scales. Data about each person, place or thing is stored in a record, which is simply a collection of key/value pairs. And you can create links among records to easily model all the relationships within your data.

Schemaless

Since Concourse makes very few assumptions about data, it integrates with your application seamlessly and never needs a translator (goodbye object-relational impedance mismatch)! You never have to declare structure up front–no schema, tables, or indexes–or specify value types because Concourse is smart enough to figure it out. Concourse dynamically adapts to your application so that you can focus on building value without having to drag the database along.

Search

Concourse supports rich full text search right out the box, so you don't need to deploy an external search server. Data is automatically indexed and searchable in real-time without ever diminishing write performance. In Concourse, you can always perform as-you-type searches that match full or partial terms.

Overview

System Requirements
  • At least 256 MB of available memory
  • Linux or OS X
  • Java 1.8+
Versioning

Concourse will be maintained under the Semantic Versioning guidelines such that release versions will be formatted as <major>.<minor>.<patch> where

  • breaking backward compatibility bumps the major,
  • new additions while maintaining backward compatibility bumps the minor, and
  • bug fixes or miscellaneous changes bumps the patch.
Additional Resources

Contributing

Read the contributing guidelines to learn how to get involved in the community. We value and welcome constructive contributions from anyone, regardless of skill level :)

Mailing Lists

Credits

Author
License

Copyright © 2013-2024 Cinchapi Inc.

Concourse is released under the Apache License, Version 2.0. For more information see LICENSE, which is included with this package.

concourse's People

Contributors

adisrini avatar aunyks avatar briannafugate408 avatar chandresh-pancholi avatar ckirksey3 avatar dearyhud avatar dubex avatar gitter-badger avatar hcuffy avatar hyinnn avatar iamutkarshtiwari avatar jamalmoir avatar javierlores avatar jaypatel4444 avatar jonathanmarvens avatar jtnelson avatar kylycht avatar larry-the-tennis-ball avatar lhcopetti avatar raghavbabu avatar tylerwbrown avatar vijayk2019 avatar vishuv avatar vrnithinkumar avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

concourse's Issues

Incorrect results when querying for data with an integer while stored type is a long

concourse.add("foo", 1L, 1);
concourse.find(Criteria.where().key("foo").operator(Operator.EQUALS).value(1)); // empty but expect [1]
concourse.find(Criteria.where().key("foo").operator(Operator.EQUALS).value(1L)); // empty but expect [1]

The data is being force added as a Long, but the query interprets the value as an int, regardless of whether we use 1 or 1L

Initializer should not run when installing. It should run when the server successfully starts for the first time

Right now the initialization script for the upgrade framework runs as part of the server installation process. The purpose of this script is to place the schema version files in the data directories so it should not run before the user has had a chance to modify concourse.prefs to set the appropriate data directories.

The correct way to do this is for ConcourseServer to check the data directories for the schema files. If they don't exist, then ConcourseServer should spawn a separate JVM to run initialization.

Additionally, instead of putting this logic in ConcourseServer.java, we can add it to the concourse ctrl script to check for the presence of the schema files and run the initializer if necessary. I don't like this approach as much though because it requires the implementation of more one off pref parsing. Nevertheless it does have the benefit of making the JVM spawning/displaying console logging easier and more natural for the user.

Calling stage() when already in a transaction should be a no-op

When writing code that uses transactions, applications need to keep track of whether the client they are using is in a transaction so that they don't accidentally start a new one before committing.

public void methodA() {
    concourse.stage();
    try {
        // do some work
        concourse.commit();
    }
    catch(TransactionException e) {
        concourse.abort();
    }
}

public void methodB() {
    concourse.stage();
    try {
        // do some work
        methodA();
        concourse.commit();
    }
    catch(TransactionException e) {
        concourse.abort();
    }
}

In the example above, the work that methodA() is done when called from methodB is an extension of the work methodB is doing, so the fact that methodB is already in a transaction should already be considered by the client when methodA is called

I can't think of a reason why the behaviour of calling stage when already in a transaction is to create a new transaction. I think its best to simply make it a no-op and continue with the current transaction, if it exists. This should be a small change.

Each client should just to see if the transaction token is not null, and if that is the case just return immediately.

We could implement this on the server if we edited the thrift API to take a transaction token in the stage method.

Specify size of collections, where possible, in Concourse Server

For the get/select records (and maybe others) we should specify the size of the maps we're using where possible. This means replacing stuff like

Map<Long, Map<String, Set<TObject>>> result = Maps.newLinkedHashMap();

with

Map<Long, Map<String, Set<TObject>>> result = TMaps.newLinkedHashMapWithCapacity(records.size());

Same thing for Sets, if possible.

Column order for data tables should match input list in CaSH

[default/cash]$ select(["name", "salary", "title"], "role = Director and salary > 10")
Returned '
+--------------------------------------------------------------------+
| Record           | title                     | name       | salary |
+--------------------------------------------------------------------+
| 1448732216678007 | [Director of Engineering] | [Jane Doe] | [20.0] |
+--------------------------------------------------------------------+
' in 0.011 sec

The order of the columns should match the order of the input list. If that isn't possible, then it should be alphabetical.

Order of values returned in select seems to be incorrect

[default/cash]$ add "baz", 10, 1
Returned 'true' in 0.011 sec

[default/cash]$ add "baz", 9, 1
Returned 'true' in 0.012 sec

[default/cash]$ add "baz", 9, 2
Returned 'true' in 0.01 sec

[default/cash]$ add "baz", 10, 2
Returned 'true' in 0.01 sec

[default/cash]$ select([1L, 2L])
Returned '
+------------------+
| Record | baz     |
+------------------+
| 1      | [9, 10] |
| 2      | [9, 10] |
+------------------+
' in 0.027 sec

I thought the order of the values was supposed to be insertion order? but it seems to place them in sort order? Not sure if this is incorrect or not, but we should figure out.

Interesting enough, it always seems to display links in descending order:

+---------------------------------------------------------------------------------------------------------------+
| Record           | bar | baz         | foo        | company    | direct_reports | name          | title | pi  | 
+---------------------------------------------------------------------------------------------------------------+
| 1447234653927000 | [1] | [true, @50] | [bar, baz] | null       | null           | null          | null  | null | 
| 1447234653927006 | null | null        | null       | [Cinchapi] | [@2, @1]       | [Jeff Nelson] | [CEO] | null | 
| 1447234653928002 | null | null        | null       | null       | null           | null          | null  | [3] | 
+---------------------------------------------------------------------------------------------------------------+

SearchBlockText.testMightContainLocatorKeyValue fails

TEST FAILURE in testMightContainLocatorKeyValue: null

---
locator = eqcicldw12dsowa7it4vi0pnqgewxci4c3ihyzf (org.cinchapi.concourse.server.model.Text)
value = w jvnwa8xofm6asavrgpyxpk1mbgah7slcaookolqo fpa3g5 5csjly (org.cinchapi.concourse.server.model.Value)
term = jvnwa8xofm6asavrgpyxpk1mbgah7slcaookolqo (org.cinchapi.concourse.server.model.Text)
record = 52259011321627880 (org.cinchapi.concourse.server.model.PrimaryKey)

Use -- skip-integration flag when installing ManagedConcourseServer

The ete test framework should use the skip-integration flag when installing the managed concourse server. Right now, the framework doesn't use the flag and employs a hack where it kills the installer process after 1 second in order to avoid hanging on waiting for the admin password.

Its possible the logic will need to check if the version >= 0.5.0 before trying to use the flag.

Add shutdown lock to ConnectionPool

ConnectionPool registers its own shutdown hook in order to ensure that the connections are released when the JVM dies. This behaviour isn't well documented and some well to do user might register their own shutdown hook to do the same.

Shutdown hooks in Java are run concurrently, so there may be a case of two threads trying to shutdown the pool at the same time.

The easiest fix for this should to change ConnectionPool#353 to

if(open.compareAndSet(true, false)){

in order to ensure that the check is atomic and two threads won't miss updates from each other.

Import framework improvements

  1. Better documentation
  2. Move general logic in LineBasedImporter to Importer base class (e.g. transformValue(), validateFileFormat(), etc).
    ...For transformValue it'll have to be able to return something more generic than a JsonElement

Import framework additional transformer options

  • allow transformer on command line of import cli by providing the path to a groovy script
  • allow transformer on command line of import cli by providing the fully qualified class name of a java file and ensuring that the file is on the classpath
  • allow specifying multiple transformers?

Error when inserting Link contained in java.util.Map

Gson does some special conversion for the built in java map that causes it to serialize Link as a JSON object.

Repro:

   @Test
    public void testMapWithLinkObjectToJson(){
        Map<String, Object> data = Maps.newHashMap();
        data.put("foo", Link.to(1));
        String json = Convert.mapToJson(data);
        Assert.assertEquals("{\"foo\":\"@1\"}", json);

    }

error:

org.junit.ComparisonFailure: expected:<{"foo":["@1"]}> but was:<{"foo":[{"record":1}]}>
    at org.junit.Assert.assertEquals(Assert.java:115)
    at org.junit.Assert.assertEquals(Assert.java:144)
    at com.cinchapi.concourse.util.ConvertTest.testMapWithLinkObjectToJson(ConvertTest.java:637)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
    at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
    at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)

Add LICENSE and NOTICE files to driver dists

The Apache License from the root should be copied into each of the license dists. For java, this means putting it in the META-INF folder of the jar.

The NOTICE files for the drivers will probably be different than that which is included for the server, so each of the project directories should have a separate notice file, if necessary.

Though, it might not be necessary to include the NOTICE or the jars since dependencies are not being bundled.

Cannot insert Map from Groovy

def data = [
    ['name':'John Doe',
    'department': 'Engineering',
    'title': 'Senior Software Engineer',
    'role': 'Software Engineer - Backend',
    'manager': Link.toWhere('title = Director of Engineering'),
    'salary': 10.00,
    'location': 'Atlanta',
    'exempt': true
    ]
]

def concourse = Concourse.connect();
println(concourse.select(concourse.insert(data)))

This yields

Caught: java.lang.ClassCastException: java.util.LinkedHashMap cannot be cast to com.google.common.collect.Multimap
java.lang.ClassCastException: java.util.LinkedHashMap cannot be cast to com.google.common.collect.Multimap
    at com.cinchapi.concourse.util.Convert.mapsToJson(Convert.java:207)
    at com.cinchapi.concourse.Concourse.insert(Concourse.java:1314)
    at com.cinchapi.concourse.Concourse$insert$0.call(Unknown Source)
    at quickstart.run(quickstart.groovy:20)

which happens because the Concourse API methods expect Multimap.

Test php-driver against PHP 5.6+

Use vagrant to setup an env with PHP 5.6 to checkout the code and run the unit tests. This is necessary to ensure that the logic for handling 64-bit pack/unpack works across versions.

Use singleton's for CaSH exceptions that merely indicate state changes

  1. Exceptions are expensive to create. For whatever reason, we use Exceptions to control program flow in CaSH (though it kind of makes sense to do this in a REPL, so forgive the departure from convention). We don't need to create a new Exception object every time for things that are merely there to control flow. We should create a singleton, similar to what we do for BufferCapacityException and throw that instead.

I think we can do this for

  • NewLineRequest
  • MultiLineRequest - not sure this exception needs to take in a message?
  • ExitRequest

But honestly, even for ones that do have state (i.e. a useful message and other attributes), we could use a singleton and just change the state whenever the exception needs to be thrown since CaSH is single threade

Order of params in link method is unintuitive

The add, set and remove methods have a signature with order key, value, record
The link and unlink methods have a signature with order key, source, destination

it would probably be more intuitive to have the link signature be key, destination, source

Cash should automatically convert method names in underscore case to camelcase

[default/cash]$ find_or_add "name", "jeff"
ERROR: No signature of method: org.cinchapi.concourse.Concourse$Client.find_or_add() is applicable for argument types: (java.lang.String, java.lang.String) values: [name, jeff]
Possible solutions: findOrAdd(java.lang.String, java.lang.Object), findOrAdd(java.lang.String, java.lang.Object)

[default/cash]$ findOrAdd "name", "jeff"

In CaSH, if compilation fails because of "No signature of method" and the method name has underscores, then convert the method name to camel case and try again

Add jsonify method

In the doJsonify method, I didn't know if I should've used StringBuilder or StringBuffer.

Add --header option to import CLI

Add an option for the user to specify the header on the command line. This will help in cases when the header is not provided in the file but the user doesn't want to write custom code to provide the header.

In the implementation, ImportCli should set the header in each importer. Each importer should also check the first line of input against the header that was provided. If the first line of input is identical to the header, that line should be skipped during import. This is meant to help in cases where multiple files are being imported, but only one of them has the header. In that case, the user can simply specify the header on the command line.

Would be nice if concourse compiled with Java 8

Has a few errors right now:

Users/curtis/git/concourse/concourse-server/src/main/java/jsr166e/ConcurrentHashMapV8.java:4663: error: spliterator() in KeySetView cannot implement spliterator() in Set
        public ConcurrentHashMapSpliterator<K> spliterator() {
                                               ^
  return type ConcurrentHashMapSpliterator<K> is not compatible with Spliterator<K>
  where K,E are type-variables:
    K extends Object declared in class KeySetView
    E extends Object declared in interface Set
/Users/curtis/git/concourse/concourse-server/src/main/java/jsr166e/ConcurrentHashMapV8.java:4729: error: spliterator() in ValuesView cannot implement spliterator() in Collection
        public ConcurrentHashMapSpliterator<V> spliterator() {
                                               ^
  return type ConcurrentHashMapSpliterator<V> is not compatible with Spliterator<V>
  where V,E are type-variables:
    V extends Object declared in class ValuesView
    E extends Object declared in interface Collection
/Users/curtis/git/concourse/concourse-server/src/main/java/jsr166e/ConcurrentHashMapV8.java:4824: error: spliterator() in EntrySetView cannot implement spliterator() in Set
        public ConcurrentHashMapSpliterator<Map.Entry<K, V>> spliterator() {
                                                             ^
  return type ConcurrentHashMapSpliterator<Entry<K,V>> is not compatible with Spliterator<Entry<K,V>>
  where K,V,E are type-variables:
    K extends Object declared in class EntrySetView
    V extends Object declared in class EntrySetView
    E extends Object declared in interface Set

Remove unnecessary nested exception handling in CaSH

            catch (Exception e) {
                if(e.getCause() instanceof TTransportException) {
                    throw new ProgramCrash(e.getMessage());
                }
                else if(e.getCause() instanceof SecurityException) {
                    throw new ProgramCrash(
                            "A security change has occurred and your "
                                    + "session cannot continue");
                }
                else if(e instanceof CompilationFailedException) {
                    throw new MultiLineRequest(e.getMessage());
                }
                else if(e instanceof MissingMethodException
                        && script != null
                        && e.getMessage().contains(
                                "No signature of method: ConcourseShell.")) {
                    String method = e.getMessage().split("ConcourseShell.")[1]
                            .split("\\(")[0];
                    input = input.replaceAll(method, "ext." + method);
                    return evaluate(input);
                }
                else {
                    String message = e.getCause() instanceof ParseException ? e
                            .getCause().getMessage() : e.getMessage();
                    throw new EvaluationException("ERROR: " + message);
                }
            }

All the places where we do e instance of ... should just be caught directly and handled in a separate block.

Composer.phar not working

Fatal error: Uncaught exception 'PharException' with message 'phar "/Users/jnelson/Dropbox/code/portland/concourse/concourse-driver-php/composer.phar" SHA1 signature could not be verified: broken signature' in /Users/jnelson/Dropbox/code/portland/concourse/concourse-driver-php/composer.phar:23
Stack trace:
#0 /Users/jnelson/Dropbox/code/portland/concourse/concourse-driver-php/composer.phar(23): Phar::mapPhar('composer.phar')
#1 {main}
  thrown in /Users/jnelson/Dropbox/code/portland/concourse/concourse-driver-php/composer.phar on line 23

CasH and other CLIs don't handle concourse_client.prefs correctly if values are non-default

  1. Create a new user named "foo" with a password that is not "admin"
  2. Add a concourse_client.prefs file to the user's home directory
username = foo
  1. Try to connect to concourse shell
$ concourse shell
ERROR: Invalid username/password combination.

The issue here is that the logic for reading the prefs file always returns the default if its not present so it is returning "admin" as the password.

Should transform value sets lazily

Instead of

                 Map<Long, Set<T>> pretty = PrettyLinkedHashMap
                            .newPrettyLinkedHashMap("Record", key);
                    for (Entry<Long, Set<TObject>> entry : raw.entrySet()) {
                        pretty.put(entry.getKey(), Transformers.transformSet(
                                entry.getValue(),
                                Conversions.<T> thriftToJavaCasted()));
                    }

should be doing something like

Map<Long, Set<T>> transformed = Maps.transformValues(raw,
                            TransformFunctions.set(Conversions
                                    .<T> thriftToJavaCasted()));

assuming TransformFunctions.set looks like

public static <V1, V2> Function<Set<V1>, Set<V2>> set(
            final Function<V1, V2> function) {
        return new Function<Set<V1>, Set<V2>>() {

            @Override
            public Set<V2> apply(Set<V1> input) {
                return Transformers.transformSet(input, function);
            }

        };
    }

Only challenge here is how do we transformed to be pretty lazily?

Change all the ConcourseServer method specs to take a boolean for transaction routing instead of a TransactionToken.

We had added timestamp with each TransactionToken precisely because we wanted to differentiate among different transactions that belonged to the same client session. However, it turns out that isn't really necessary because a client session can only have one transaction open at a time.

In actuality, we don't even need TransactionTokens anyway. We can just associate the AccessToken with the Transaction object and that should be sufficient. And, instead of sending over a possibly null TransactionToken for each RPC call to ConcourseServer, we could just send over a boolean that indicates whether the call should be routed to a transaction or not.

Don't get maven repo from external file

We depend on the .maven file to tell Gradle the correct repo for archive uploading. Why don't we just add logic that checks the correct conditions in code:

maven {
          credentials {
            username "admin"
            password "password"
          }
          if(project.version.endsWith('-SNAPSHOT')) {
            url "http://xxx"
          } else {
            url "http://xxx"
          }
        }

Allow querying on the record id

It'd be nice to be able to query on the record id

where foo = bar and $id$ != 123

This should be allowable since there is a reserved identifier for referring to the id of a record.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.