derive
Transforms data before it reaches the next operator or mark. The function receives the current data group and returns a new one.
gf.chart(seafood, { axes: true })
.flow(
gf.derive((d) => d.filter((row) => row.species === "Salmon")),
gf.spread({ by: "lake", dir: "x" })
)
.mark(gf.rect({ h: "count", fill: "steelblue" }))
.render(root, { w: 400, h: 250 });Signature
derive(fn);
derive(fn, { schema });Parameters
fn (required, (rows: T[]) => U[] | Promise<U[]>) receives the rows at this point in the flow (inside .flow(), the current group's rows) and returns the list of rows the rest of the pipeline sees. It returns a list even for one row ((d) => [d[0]]). It may be async. The options:
Transforms the data with a function, derive(fn). A function does not serialize: the IR carries a Python bridge handle in its place.
| Option | Type | Default | Description |
|---|---|---|---|
schema | Record<string, any> | Column types of the result, keyed by column name, as in a chart's schema, e.g. Unit({ domain: Domain.discrete(levels), traits: [Trait.ordered()] }) or Unit.time(). They override the types the result keeps from its input or infers, and convert values (an ISO string in a time column becomes an instant). |
schema takes the same column types as chart's schema.
Column types of the result
Each column of the returned rows is typed like chart data:
- A column whose values still fit the type it had in the input keeps that type. A time column of unchanged instants stays a time, and an ordered column whose values are still text or numbers keeps the order, so a value outside its levels is an error where the order is used.
- A column of
Datevalues is a time in UTC. - Any other column has no type. A
derivethat rewrites a date to"Mar"makes the column plain text. - A
schemaentry overrides these for its column and converts the values as a chart's schema does: an ISO 8601 string in aUnit.time()column becomes an instant.
A result that is one object, not an array, is typed as one row the same way. A column of Date values that had a time type in the input keeps it, time zone included. Fitting reads the values, not what they mean: a derive that turns a time column into plain numbers, such as years, keeps the time type, because any number is a valid instant (#1089). Give the column its type with derive(fn, { schema }).
filter carries the input's types over as they are, since its rows are the input's rows.
// Monthly highs, months in calendar order
derive(monthlyHighs, {
schema: {
month: Unit({ domain: Domain.discrete(MONTHS), traits: [Trait.ordered()] }),
},
});Examples
// Filter before spreading
.flow(
derive(d => d.filter(row => row.year === 2020)),
spread({ by: "category", dir: "x" })
)
// Compute a per-group sum (after spread, d is scoped to one group)
.flow(
spread({ by: "category", dir: "x" }),
derive(d => [{ ...d[0], total: sumBy(d, "value") }])
)
// Reshape wide-to-long
.flow(
derive(d => d.flatMap(row => [
{ ...row, measure: "a", value: row.a },
{ ...row, measure: "b", value: row.b },
]))
)Data helpers
Helpers for the function derive takes:
normalize(rows, field): each row withfielddivided by the column's total.repeat(row, field): the row,row[field]times.quantileRows(rows, { x, density, n }):nrows at the quantiles of a density curve, one at each probability(i + 0.5) / n.rowsis a grid of one curve, each row a positionxand thedensitythere, in any order, with no two rows at one position (call it after the split into curves). The density is integrated by the trapezoid rule. Each row made holdsxand every column whose value is the same in all ofrows(a group'smonth).Bin.quantilesandfield(x).median()over the result read the curve instead of the grid. It stands in for first-class distributions (#980), which will replace it.
// The quantiles of a drawn density, cut into ten equal-mass cells
.flow(
derive((rows) => quantileRows(rows, { x: "x", density: "pdf", n: 1000 })),
partition({ by: field("x").bin(Bin.quantiles(10)), dir: "x" })
)
.mark(circle({ cx: field("x").median(), r: 3 }))Units: keeping axes shared across a transform
A channel that encodes a column carries the column's unit. GoFish uses units to decide when two axes may share a scale. A column with no declared unit has an unknown unit, which shares an axis with any column, so the columns a derive computes (lo, hi, a box plot's quartiles) share an axis with no annotation. Two different declared units on one axis are an error. See Unit.
A column's type (its declared quantity and unit,
HasQuantityandHasUnit) rides the data array, so a column the input declares keeps it throughderive.An arbitrary
derivecan declare the unit of a column it makes with itsschema:tsderive(quartiles, { schema: { lo: Unit("USD"), hi: Unit("USD") }, });
An axis is titled by the quantities of its columns (each column's name, unless its schema declares a quantity), then a declared unit in parentheses: "Pay (USD)". See Schema.
If an axis combines two different units, you have two remedies:
- If the units really are the same, declare the same unit for both columns.
- If the units really differ, give the inner chart an explicit
w/hso it becomes a self-contained scale region and never shares that axis.
