Skip to content

Introduction to Concourse#

Concourse is an open source self-tuning database warehouse that provides strong consistency, decentralized coordination, and optimistic availability. Concourse takes the best ideas from SQL and NoSQL databases to provide an intuitive and scalable platform for both transaction and analytic workflows.

Document-Graph Database#

Concourse is a document-graph database.

A record in Concourse is a schemaless document, composed of a unique ID and one or more fields. Each field is labeled with a key and may contain one or more distinct primitive values. Additionally, fields may contain links to other records, which facilitates modeling a rich graph of descriptive relationships.

Concourse records are similar to linked JSON objects.

The document-graph data model is ideal for development:

  • Intuitively mirrors the way that developers think about data.
  • Makes few assumptions and adapts to application changes without translation (goodbye object-relational impedance mismatch).
  • Models complex relationships with referential integrity while avoiding hacks.

Automatic Indexing#

Concourse automatically indexes data for search and analytics while guaranteeing constant time writes that are incredibly fast. This eliminates the need to plan queries in advance because Concourse fully supports ad-hoc, range, and predicate queries and caches frequently requested data for optimal performance.

Version Control#

Concourse automatically tracks changes to data — like Git does for source code — giving you the power to query data from the past, audit changes on the fly, and revert to previous states without downtime. Version control in Concourse makes it easy to build applications that leverage definitive data across time.

ACID Transactions#

Concourse provides serializable distributed transactions. Internally, dynamic resource allocation and just-in-time locking ensure that transactions have both the highest performance and strongest consistency possible. So there is no need to guess when your data will eventually become consistent. When Concourse responds to a query, you can trust the results immediately.

By default, each change is autocommitted and written to the database immediately. However, you can explicitly start a transaction to control when a group of changes are atomically committed or aborted using the stage functionality.

  • Start a transaction within CaSH:
1
[default/cash]$ stage
  • Use a transaction to atomically transfer $50 from one account to another in Java:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
concourse.stage();
try {
  int balance1 = concourse.get("balance", 1);
  int balance2 = concourse.get("balance", 2);
  concourse.set("balance", balance1 + 50, 1);
  concourse.set("balance", balance2 - 50, 2);
  concourse.commit();
}
catch (TransactionException e) {
  concourse.abort();
}
  • The same example using the lambda syntax:
1
2
3
4
5
6
concourse.stage(() -> {
  int balance1 = concourse.get("balance", 1);
  int balance2 = concourse.get("balance", 2);
  concourse.set("balance", balance1 + 50, 1);
  concourse.set("balance", balance2 - 50, 2);
});

Command API#

Concourse provides a type-safe Command API for building and executing operations programmatically:

1
2
concourse.exec(
    Command.to().select("name", "age").from(1));

Commands can also be expressed in CCL syntax directly in CaSH:

1
[default/cash]$ select name, age from 1

Aggregations#

Concourse includes built-in aggregation functions for performing calculations across data:

1
2
Number avg = concourse.calculate().average(
    "salary", "department = Engineering");

Concourse automatically indexes all string data for full-text search:

1
Set<Long> records = concourse.search("bio", "distributed");

Graph#

Concourse supports graph operations through links and navigation keys:

1
2
3
concourse.link("employer", 100, 1);
Map<Long, Set<Object>> names =
    concourse.navigate("employer.name", 1);

Plugins#

Concourse can be extended by plugins that run in isolated processes and have full access to the database.

Environments#

Concourse separates records into environments, which are similar to databases, keyspaces, or schemas in other database systems. Beyond that, Concourse does not impose any other organization such as tables or collections.

You can specify the desired environment when connecting to Concourse. If no environment is specified, the default_environment defined in the concourse.yaml configuration file is used.

  • Connect to the production environment using CaSH:
1
concourse shell -e production
  • Connect to the production environment using the Java driver:
1
2
3
Concourse concourse = Concourse.at()
    .environment("production")
    .connect();

Warning

It is not possible to interact with data in multiple environments simultaneously from a single connection.

Create an Environment#

If an environment does not exist, Concourse creates it when you first store data within the environment. As such, you can connect to a non-existent environment and perform standard operations.