Skip to content

Time Travel#

Concourse automatically tracks every change to every piece of data, creating a complete version history. This enables you to query and read data as it existed at any point in the past.

Temporal Queries vs Temporal Reads#

There are two distinct ways to access historical data:

Temporal Queries#

A temporal query evaluates its selection criteria based on the version of data at a previous timestamp. The query returns the records that would have matched the criteria at that point in time.

1
2
3
4
5
// Java
// Find records where age > 30 as of last month
Set<Long> records = concourse.find(
    "age > 30",
    Timestamp.fromString("last month"));

Temporal Reads#

A temporal read retrieves the version of the data that existed at a previous timestamp. The read returns what the data looked like at that point in time.

1
2
3
4
// Java
// Get the name that was stored in record 1 last month
Object name = concourse.get("name", 1,
    Timestamp.fromString("last month"));

Combining Both#

You can combine temporal queries and temporal reads in a single operation:

1
2
3
4
5
6
// Java
// Select data from records matching criteria, both
// evaluated at the same historical timestamp
Map<Long, Map<String, Set<Object>>> data =
    concourse.select("age > 30",
        Timestamp.fromString("last month"));

Timestamps#

Most Concourse methods accept an optional Timestamp parameter for time travel.

Creating Timestamps#

1
2
3
4
5
6
// Java
Timestamp now = Timestamp.now();
Timestamp specific =
    Timestamp.fromMicros(1609459200000000L);
Timestamp fromDate =
    Timestamp.fromJoda(new DateTime(2025, 1, 1, 0, 0));

Natural Language Timestamps#

Concourse supports natural language time expressions, which are especially convenient in the CaSH shell:

1
2
3
4
5
6
// CaSH
get "name" from 1 at "yesterday"
get "name" from 1 at "3 hours ago"
get "name" from 1 at "last Tuesday"
get "name" from 1 at "January 1, 2025"
get "name" from 1 at "2 weeks ago"

1
2
3
// Java
Timestamp ts = Timestamp.fromString("3 days ago");
Object name = concourse.get("name", 1, ts);

Audit#

The audit method returns a complete revision history for a record or a specific field. Each entry maps a commit timestamp to a list of human-readable descriptions of the changes made.

Audit an Entire Record#

1
2
3
// Java
Map<Timestamp, List<String>> history =
    concourse.audit(1);

1
2
// CaSH
audit 1

Audit a Specific Key#

1
2
3
// Java
Map<Timestamp, List<String>> history =
    concourse.audit("name", 1);

1
2
// CaSH
audit "name", 1

Audit a Time Range#

Restrict the audit to changes within a specific time window (start inclusive, end exclusive):

1
2
3
4
5
6
7
// Java
Timestamp start =
    Timestamp.fromString("January 1, 2025");
Timestamp end =
    Timestamp.fromString("February 1, 2025");
Map<Timestamp, List<String>> history =
    concourse.audit(1, start, end);

Chronicle#

The chronicle method returns a time series of the values stored for a key in a record after every change. Unlike audit, which describes what changed, chronicle shows the state of the field after each change.

1
2
3
4
5
// Java
Map<Timestamp, Set<Object>> timeline =
    concourse.chronicle("name", 1);
// e.g., {T1 -> ["Jeff"], T2 -> ["Jeff", "Jeffrey"],
//         T3 -> ["Jeffrey"]}

1
2
// CaSH
chronicle "name", 1

Chronicle a Time Range#

1
2
3
4
// Java
Map<Timestamp, Set<Object>> timeline =
    concourse.chronicle("name", 1,
        Timestamp.fromString("last month"));

Diff#

The diff method returns the net changes between two points in time. Unlike audit, which records every individual change, diff computes the minimal set of additions and removals needed to transition from the start state to the end state.

Diff a Field#

1
2
3
4
5
// Java
Map<Diff, Set<Object>> changes =
    concourse.diff("name", 1,
        Timestamp.fromString("last week"));
// e.g., {ADDED -> ["Jeffrey"], REMOVED -> ["Jeff"]}

Diff an Entire Record#

1
2
3
4
// Java
Map<String, Map<Diff, Set<Object>>> changes =
    concourse.diff(1,
        Timestamp.fromString("last week"));

Diff a Key Across All Records#

1
2
3
4
5
// Java
Map<Object, Map<Diff, Set<Long>>> changes =
    concourse.diff("status",
        Timestamp.fromString("last week"));
// Shows which values were added/removed and in which records

Diff Between Two Timestamps#

All diff variants accept a start and optional end timestamp. If only a start is provided, the diff covers from the start to the present.

1
2
3
4
5
6
7
// Java
Timestamp start =
    Timestamp.fromString("January 1, 2025");
Timestamp end =
    Timestamp.fromString("February 1, 2025");
Map<Diff, Set<Object>> changes =
    concourse.diff("name", 1, start, end);

Revert#

The revert method atomically restores a key in a record to its state at a previous timestamp. Concourse computes the necessary adds and removes to recreate the historical state, then applies them as new revisions. The original history is preserved.

1
2
3
// Java
concourse.revert("name", 1,
    Timestamp.fromString("last week"));

1
2
// CaSH
revert "name", 1, "last week"

Revert Multiple Keys#

1
2
3
4
// Java
concourse.revert(
    Lists.newArrayList("name", "email"), 1,
    Timestamp.fromString("yesterday"));

Revert Across Multiple Records#

1
2
3
4
// Java
concourse.revert("status",
    Lists.newArrayList(1L, 2L, 3L),
    Timestamp.fromString("last week"));

Revert creates new revisions

Reverting does not delete history. Instead, it creates new revisions that undo the changes made since the target timestamp. The full audit trail, including the revert operations themselves, is preserved.

Historical Reads#

All standard read methods support a Timestamp parameter for historical lookups:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
// Java
// Historical select
Map<String, Set<Object>> data = concourse.select(1,
    Timestamp.fromString("yesterday"));

// Historical get
Object name = concourse.get("name", 1,
    Timestamp.fromString("yesterday"));

// Historical browse
Map<Object, Set<Long>> index = concourse.browse(
    "status",
    Timestamp.fromString("last month"));

// Historical describe
Set<String> keys = concourse.describe(1,
    Timestamp.fromString("last year"));

// Historical verify
boolean existed = concourse.verify("name", "Jeff", 1,
    Timestamp.fromString("2 days ago"));

Historical Navigation#

Navigation and trace methods also support temporal lookups:

1
2
3
4
5
6
7
8
// Java
Map<Long, Set<Object>> results =
    concourse.navigate("employer.name", 1,
        Timestamp.fromString("last quarter"));

Map<String, Set<Long>> incoming =
    concourse.trace(100,
        Timestamp.fromString("6 months ago"));