Skip to content

Graph#

Concourse’s document-graph data model lets you store relationships between records and traverse them efficiently. Relationships are represented as Links — directional pointers from one record to another.

Linking Records#

Use the link method to create a directed relationship from a source record to a destination record via a named key.

1
2
3
// Java
// Create a link: record 1 --employer--> record 100
concourse.link("employer", 100, 1);

1
2
// CaSH
link "employer", 100, 1

Links are directional. The example above creates a relationship from record 1 to record 100, but record 100 has no automatic back-reference to record 1.

1
2
3
// Java
concourse.link("friends",
    Lists.newArrayList(2L, 3L, 4L), 1);

Use unlink to remove a relationship:

1
2
// Java
concourse.unlink("employer", 100, 1);

1
2
// CaSH
unlink "employer", 100, 1

You can query for records that link to a specific destination using the LINKS_TO operator:

1
employer lnk2 @100

1
2
3
// Java
Set<Long> employees = concourse.find(
    "employer", Operator.LINKS_TO, Link.to(100));

This returns all records whose employer key contains a link to record 100.

Navigation lets you traverse links and read data from linked records using dot-separated key paths called navigation keys.

A navigation key is a dot-separated path where each segment before the last is a key containing links, and the final segment is the key to read from the destination record.

1
employer.name

This means: follow the employer link, then read the name key from the linked record.

Using navigate()#

The navigate method traverses navigation keys and returns data from the destination records.

1
2
3
4
// Java
// Get the name of the employer linked from record 1
Map<Long, Set<Object>> results =
    concourse.navigate("employer.name", 1);

1
2
// CaSH
navigate "employer.name" from 1

Multi-Hop Navigation#

Navigation keys can span multiple hops:

1
employer.ceo.name

This traverses two links: first employer, then ceo, and finally reads name from the destination.

1
2
3
// Java
Map<Long, Set<Object>> results =
    concourse.navigate("employer.ceo.name", 1);

1
2
3
4
5
6
// Java
Map<Long, Map<String, Set<Object>>> results =
    concourse.navigate(
        Lists.newArrayList(
            "employer.name", "employer.city"),
        1);

1
2
3
4
// Java
Map<Long, Set<Object>> results =
    concourse.navigate("employer.name",
        Lists.newArrayList(1L, 2L, 3L));

Navigate from all records matching a query:

1
2
3
4
// Java
Map<Long, Set<Object>> results =
    concourse.navigate("employer.name",
        "department = Engineering");

Historical Navigation#

1
2
3
4
// Java
Map<Long, Set<Object>> results =
    concourse.navigate("employer.name", 1,
        Timestamp.fromString("last month"));

Per-Stop Brackets#

Each stop of a navigation path can carry its own [<timestamp>] bracket independently. The bracket pins that stop’s read at the specified moment, while unbracketed stops default to the operation-level timestamp (or the present moment if none is supplied):

1
2
3
4
5
6
-- Pin only the link traversal; read each destination's
-- current salary
select manager["last week"].salary from 1

-- Pin both the link traversal and the destination read
select manager["last week"].salary["start of year"] from 1

Per-stop brackets are useful when the graph shape and the destination values belong to different points in time — for example, “who reported to me last week, and what do they earn today?” See Per-Key Timestamps for the full precedence rule.

Navigation keys can be used directly in CCL conditions to filter records based on data in linked records:

1
employer.name = "Cinchapi"

1
2
3
// Java
Set<Long> records = concourse.find(
    "employer.name = \"Cinchapi\"");

This finds all records whose linked employer record has a name of "Cinchapi". Navigation keys work with all CCL operators:

1
2
3
employer.founded > 2010
manager.department contains "Engineering"
employer.address.city = "Atlanta"

Transitive Navigation#

When a field contains self-referential links — for example, a children key whose links point to more records that also have children — the depth of the graph is data-dependent. You can follow those links recursively by appending a * suffix to any stop in a navigation path.

1
2
3
children*.name
parent*.title
manager*.email

The * tells the server to expand that stop with a breadth-first search: start from the links at the current stop, follow them to their destinations, and if any destination record also has outgoing links on the same field, follow those too, until no new records are discovered. The entire traversal runs server-side in a single RPC.

Cycles and Termination#

Transitive traversal automatically deduplicates records as the BFS frontier expands, so cyclic graphs (e.g., a record that eventually links back to an ancestor) terminate cleanly without infinite loops. The result includes each reachable record at most once.

Examples#

1
2
3
4
5
6
7
8
9
// Java
// Given record 1 → [2, 3]; 2 → [4, 5]; 5 → [6]
//   with "children" links between them, return the names of
//   every descendant in the subtree rooted at record 1
Map<String, Set<Object>> result =
    concourse.select(
        Lists.newArrayList("children*.name"), 1);
Set<Object> allDescendantNames =
    result.get("children*.name");

1
2
3
4
5
// Java
// Find every record that transitively reaches a "Bob" somewhere
// in its descendant tree
Set<Long> ancestors = concourse.find(
    "children*.name", Operator.EQUALS, "Bob");

1
2
3
// CaSH
select "children*.name" from 1
find "children*.name = Bob"

Mixed Stops in the Same Key#

A navigation path can combine transitive and non-transitive stops. Each * expands independently:

1
parent.children*.name

This follows parent once (single hop), then expands the children field transitively on the destination record, then reads name at every descendant.

Multiple transitive stops in the same key are also supported:

1
departments*.manager.reports*.email

Each * expands via its own BFS; the traversals compose in order along the path.

Supported Operations#

Transitive keys work anywhere a navigation key works:

Operation Transitive keys
select ✓
get ✓
navigate ✓
browse ✓
find ✓

If the * modifier is applied to a stop whose field does not contain links (for example, a plain string field), the BFS finds no outgoing edges to follow and the traversal terminates immediately. The result is the same as the non-transitive variant of the same key — transitive navigation degrades gracefully rather than raising an error.

Scoped Navigation#

When a record has a multi-valued link — for example, assignments pointing at several tickets, or members pointing at several users — a query that combines two conditions over the same navigation prefix can silently match records that should not qualify.

Consider a source record R with two assignments links:

1
2
3
R
├─ assignments → Ticket 1 (priority = "high", status = "open")
└─ assignments → Ticket 2 (priority = "low", status = "closed")

A query that intends to find records with a high-priority open assignment often looks like this:

1
assignments.priority = "high" and assignments.status = "open"

Plain . navigation evaluates each condition independently and intersects the resulting record sets. The first condition matches R (via Ticket 1, which has priority = "high") and the second also matches R (via Ticket 2, which has status = "open"), so R is returned — even though no single assignment has both properties.

Scoped navigation makes that “same-assignment” intent explicit with a prefix.(...) group:

1
assignments.(priority = "high" and status = "open")

Read this as: “find records that have an assignments link to a record where both priority = "high" and status = "open" hold.” The group pins every inner condition to the same destination reached via assignments, eliminating the cross-link false positive. R no longer matches, because no single linked ticket satisfies both leaves.

When to Reach for It#

Use scoped navigation whenever you combine two or more conditions (with AND) that share a navigation prefix and the prefix can be multi-valued. If the prefix only ever points at one record, the scoped and non-scoped forms return the same result, and you can safely continue using plain . navigation.

Loose (non-scoped) navigation retains its historical behavior, so existing queries are unaffected. Scoped navigation is a tightening you opt in to when you need it.

Inner Keys are Relative to the Pivot#

Inside a scope, keys are resolved from the record reached via the prefix — the pivot. A plain key references a field on the pivot:

1
assignments.(priority = "high" and status = "open")

Here priority and status both live on the assignment record.

An inner key can itself be a navigation key, which re-enters navigation from the pivot:

1
assignments.(priority = "high" and owner.team = "platform")

Here owner.team follows the assignment’s owner link to another record and reads team there. The whole scope semantically reads as: “find records with an assignment whose priority is high and whose owner is on the platform team.”

Multi-Segment and Transitive Prefixes#

The prefix is whatever you write before the .(. Multi-segment prefixes pivot at a deeper record:

1
company.departments.(name = "Engineering" and headcount > 50)

The pivot here is the department record reached via company.departments. The inner conditions must both hold at the same department.

Transitive prefixes pivot at every record in the transitive closure:

1
descendants*.(name = "Acme" and status = "active")

Any single record reachable via the transitive descendants expansion that has both name = "Acme" and status = "active" causes the source to match.

Nested Scopes#

Scopes can nest, and each nested scope rebases the pivot at its own prefix — resolved from the enclosing pivot, not from the original source:

1
2
3
4
team.(
    name = "Platform"
    and members.(role = "lead" and active = true)
)

This reads: “find records with a team link to a record named Platform that itself has at least one member who is both a lead and currently active.”

Composing With AND and OR#

A scoped group is a first-class condition, so it composes with ordinary leaves and with other scopes:

1
2
priority = "P0"
or assignments.(priority = "high" and status = "open")

1
2
tier = "enterprise"
and contacts.(role = "owner" and verified = true)

Java Driver (Criteria.scope)#

Every scope expressible in CCL is also available through the Criteria builder, mirroring the existing Criteria.group method:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
// Java
Criteria c = Criteria.where()
        .scope("assignments",
                Criteria.where()
                        .key("priority").operator(Operator.EQUALS)
                        .value("high")
                        .and()
                        .key("status").operator(Operator.EQUALS)
                        .value("open"))
        .build();
Set<Long> records = concourse.find(c);

Scopes from the builder mix freely with group, and, and or:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
// Java
Criteria c = Criteria.where()
        .key("priority").operator(Operator.EQUALS).value("P0")
        .or()
        .scope("assignments",
                Criteria.where()
                        .key("priority").operator(Operator.EQUALS)
                        .value("high")
                        .and()
                        .key("status").operator(Operator.EQUALS)
                        .value("open"))
        .build();

A scoped query follows a link. If the scope prefix has no link values at the traversal time — because the key has never been a link, because it carries only non-link values, or because its links have been removed — no record is reachable as a pivot and the result is empty. Writing a scope over a non-link or unused prefix is safe; it will not raise an error and it will not silently fall back to evaluating the inner conditions against the source record.

Timestamps in Scoped Queries#

A scoped query involves two logically distinct reads: the prefix traversal that determines which records count as pivots, and the leaf reads that evaluate each inner condition at each pivot. The two are pinned independently:

  • A bracket on the scope prefix pins prefix traversal. A[t].(...) traverses A at t. This is the only way to change the prefix-traversal time.
  • Per-leaf brackets pin their own leaf reads. They have no effect on prefix traversal.
  • Both default to the present moment when no bracket is attached. Multi-stop prefixes and nested scopes apply the rule independently at each level.

The contract over the canonical cases is:

  1. No timestamps.
1
2
A.(foo = "X" AND bar = "Y")
// A is traversed at present; foo and bar read at present.
  1. Manual uniform per-leaf timestamps.
1
2
A.(foo = "X" at t AND bar = "Y" at t)
// A is traversed at present; foo and bar read at t.

Per-leaf brackets bind only their own reads. Stamping every inner leaf at the same t does not pin the prefix — use A[t].(...) (case 6) for that.

  1. Mixed per-leaf timestamps.
1
2
3
A.(foo = "X" at t1 AND bar = "Y" at t2)
// A is traversed at present; foo reads at t1; bar reads
// at t2.
  1. Partially stamped leaves.
1
2
3
A.(foo = "X" at t AND bar = "Y")
// A is traversed at present; foo reads at t; bar reads at
// present.
  1. Bracket-pinned prefix.
1
2
A[t].(foo = "X" AND bar = "Y")
// A is traversed at t; foo and bar read at present.
  1. Bracket-pinned prefix with per-leaf brackets.
1
2
A[t3].(foo = "X" at t1 AND bar = "Y" at t2)
// A is traversed at t3; foo reads at t1; bar reads at t2.

Each bracket binds independently. To pin the entire scope at one moment, repeat the timestamp on the prefix and on every unbracketed leaf:

1
2
A[t].(foo = "X" at t AND bar = "Y" at t)
// A is traversed at t; foo and bar read at t.

Criteria#at(Timestamp) and Scoped Criteria#

Applying Criteria#at(Timestamp) to a built Criteria clobbers every inner leaf’s timestamp to the supplied value. It does not bracket the scope prefix. So a scoped criteria constructed via the builder and bound with at(t) reads each leaf at t but traverses the prefix at present:

1
2
3
4
5
6
7
8
Criteria.where()
    .scope("A", Criteria.where()
        .key("foo").operator(Operator.EQUALS).value("X")
        .and()
        .key("bar").operator(Operator.EQUALS).value("Y"))
    .build()
    .at(tPast);
// foo and bar read at tPast; A traverses at present.

To pin both the prefix traversal and the leaf reads at the same moment, embed the timestamp directly in the prefix string passed to scope(...):

1
2
3
4
5
6
7
8
Criteria.where()
    .scope("A[" + t.getMicros() + "]", Criteria.where()
        .key("foo").operator(Operator.EQUALS).value("X")
        .and()
        .key("bar").operator(Operator.EQUALS).value("Y"))
    .build()
    .at(t);
// A is traversed at t; foo and bar read at t.

Tracing References#

The trace method returns all incoming links to a record — it answers the question “which records link to this one?”

1
2
3
4
// Java
Map<String, Set<Long>> incoming =
    concourse.trace(100);
// e.g., {"employer" -> [1, 2, 3], "partner" -> [50]}

1
2
// CaSH
trace 100

The result maps each key name to the set of records that contain a link to the traced record via that key.

Trace Multiple Records#

1
2
3
// Java
Map<Long, Map<String, Set<Long>>> results =
    concourse.trace(Lists.newArrayList(100L, 200L));

Historical Trace#

1
2
3
4
// Java
Map<String, Set<Long>> incoming =
    concourse.trace(100,
        Timestamp.fromString("last week"));

Consolidating Records#

The consolidate method atomically merges data from one or more source records into a target record. Every field from each source is added to the target, the sources are cleared, and every incoming link in the database that referenced a source is rewritten to point at the target. The operation is atomic — either every source is fully consolidated and every incoming link is rewritten, or nothing changes.

1
2
3
// Java
// Merge records 200 and 300 into record 100
concourse.consolidate(100, 200, 300);

1
2
// CaSH
consolidate 100, 200, 300

Consolidation is useful when resolving duplicate records or normalizing references across a graph. Because it rewrites inbound links in place, callers that held a link to a source record continue to work — their link now resolves to the consolidated target.

Traversal Optimization#

Concourse automatically selects the most efficient traversal strategy for navigation queries. Depending on the shape of the data, it may use:

  • Forward traversal: Start from the source records and follow links forward to find destination data. This is more efficient when there are fewer source records than destination records.

  • Reverse traversal: Start from potential destination records and trace links backward to find matching sources. This is more efficient when there are fewer destination records than source records.

The optimizer chooses the strategy automatically based on data characteristics. No manual tuning is required.

For transitive keys, the same optimizer considers both strategies but biases toward reverse traversal when the first stop in the path is transitive, since a forward-expansion starting point would need to enumerate an unbounded subgraph before doing any filtering.