Dist
The Dist family holds distributions. A distribution is a value a channel reads through its members: d.density() and d.domain() are functions of t for the shape form of ribbon and line, and d.median() is one value per group for any channel. A column can hold a distribution too: kde smooths a group's values into one, and field(x) reads it through the same members.
d = Dist.normal(mean="mean", sd="sd")
chart([{"mean": 0, "sd": 1}]).mark(
ribbon(x=d.domain(), h=d.density(), fill="#4c78a8")
)Signature
Dist.normal(mean=..., sd=...)
Dist.lognormal(meanlog=..., sdlog=...)
Dist.uniform(min=..., max=...)
Dist.exponential(rate=...)
kde(column, bandwidth=None) # a derive helper
d.density()
d.cdf()
d.domain()
d.quantile(p)
d.median()
d.mean()
d.mass()
d.bin(p)The family is also a module of its own:
from gofish import Dist
from gofish.dist import normal # the same function as Dist.normalFamilies
| Member | The distribution |
|---|---|
Dist.normal(mean=..., sd=...) | The normal distribution with mean mean and standard deviation sd. |
Dist.lognormal(meanlog=..., sdlog=...) | exp of a normal with mean meanlog and standard deviation sdlog. |
Dist.uniform(min=..., max=...) | The uniform distribution from min to max. |
Dist.exponential(rate=...) | The exponential distribution with rate rate, so its mean is 1 / rate. |
Each parameter is a number, a column name, or a value expression such as field("t").mean(). A column is read over each group's rows, and it must have one value in the group. A column with several values in one group is an error. A distribution goes on the wire as data, and JS computes it, with no call into Python.
Members
| Member | Gives | Unit |
|---|---|---|
d.density() | a function of t: the mass times the probability density | the mass per the variable |
d.cdf() | a function of t: the share of the mass at or below t | a ratio |
d.domain() | a function of t: t itself, over the same interval as the density | the variable |
d.quantile(p) | the value with the share p of the mass at or below it | the variable |
d.median() | d.quantile(0.5) | the variable |
d.mean() | the mean | the variable |
d.mass() | 1 for a family, the number of values for a kernel density | a probability, or a count |
The members describe one distribution. The density's area over the whole line is the mass. The cdf is the density's integral over the mass, the quantile is the inverse of the cdf, the median is d.quantile(0.5), and the mean is the density's mean.
The functions run over the interval the density is drawn over. For a family that is its support, cut at the 0.001 and 0.999 quantiles where the support has no end: a normal runs from its 0.001 to its 0.999 quantile, a lognormal and an exponential from 0 to their 0.999 quantile. For a kernel density it runs from 3 bandwidths below the least value to 3 above the greatest, so the density tapers to near zero at both ends.
d.domain() is the interval GoFish draws the density over. It is not the distribution's support when the support is unbounded: a normal's support is every number, and its d.domain() ends at its 0.001 and 0.999 quantiles.
The variable of a family is in the unit of the column its location parameter reads (mean for normal, min for uniform), so the axis of d.domain() joins an axis in that unit. It is not that column, so it has no name, and its axis has no title unless you give one. A density is in the mass per the variable's unit, or per a difference of two of its values (a density over times is per duration). A density adds up, so densities stack.
Cells: d.bin(p)
d.bin(p) is a key for partition: the cells of p (Bin.step, Bin.thresholds or Bin.quantiles) over the interval the density is drawn over. Bin.quantiles(n) cuts at the distribution's own quantiles, so each cell holds 1 / n of the mass. In a cell, every member is the distribution restricted to the cell: the density and the domain run over the cell, d.mass() is the cell's share of the mass (a cell at an end of the interval also holds the tail past it), and d.median() is the median of the mass in the cell. A member of a distribution, such as d.density().bin(p), is not binned, and raises a ValueError.
d = Dist.normal(mean="mean", sd="sd")
partition(by=d.bin(Bin.quantiles(5)), dir="x")kde(column, bandwidth=None)
A helper for derive: derive(kde("x")) takes the rows as a sample of the column x, and gives one row whose x holds the sample's kernel density. The row also holds every other column whose value is the same in all the rows, such as a group's species after a split by it. In a flow after a split, derive runs in each group, so each group's column holds its own distribution. No rows give no row.
The density is a normal kernel on each value, with area the number of values, so the densities of several groups add up. bandwidth is the kernel's standard deviation, in the column's unit; without it, it is Silverman's rule (R's bw.nrd0), and a sample of fewer than two different values is an error that asks for one. Missing values are left out. The density is drawn from 3 bandwidths below the least value to 3 above the greatest, as seaborn's cut=3 draws it. There is no option yet to choose how far past the values the density runs, or to bound it at a hard limit, such as 0 for counts.
The column keeps its name and unit, so field("x").domain() joins the axis of x. A field of the column reads the distribution: its members are those above, and field("x").bin(Bin.quantiles(n)) is a key for partition that cuts it at its own quantiles. field("x").median() is the distribution's median, not the data's. kde runs in Python and only gathers the values; JS computes the distribution.
mass = field("Body Mass (g)")
chart(penguins).flow(
spread(by="Species", dir="x"), derive(kde("Body Mass (g)"))
).mark(ribbon(y=mass.domain(), w=mass.density(), fill="Species"))The data's own summaries
A summary of the data, such as its median, is read before the kde, while the rows are still the data. A column whose value is the same in a group's rows is kept in the kde's row. Here each violin's dot is its species' median body mass, from the rows:
from statistics import median
COL = "Body Mass (g)"
mass = field(COL)
def with_median(rows):
"""Each group's median, as a column of every row."""
m = median(r[COL] for r in rows if r[COL] is not None)
return [{**r, "median": m} for r in rows]
chart(penguins).flow(
spread(by="Species", dir="x", spacing=64),
derive(with_median),
derive(kde(COL)),
).mark(
layer(
[
stack(
[ribbon(y=mass.domain(), w=mass.density(), fill="Species")],
dir="x",
baseline=Baseline.middle(),
),
circle(cy="median", r=3, fill="white"),
]
)
)Other tools draw the same: ggplot2 4.0's geom_violin draws its quantiles from the data under a kernel density body, ggdist pairs a density estimate with the data's empirical cdf, and seaborn's violinplot draws the data's quartiles (inner="quart") on a kernel density.
As data
A distribution in a column is data, a plain dict, so it crosses the bridge as data:
{"type": "dist", "family": "kde", "params": {"values": [3750, 3800, ...], "bandwidth": 100}}
{"type": "dist", "family": "normal", "params": {"mean": 0, "sd": 1}}The first is what kde makes: the values it smooths, and the bandwidth when one is given. The second is a family with numbers for its parameters, the wire form of Dist.normal(mean=0, sd=1). A column may hold either.
