Introduction
Dioptra is a desktop program for the part of science that happens between the instrument and the paper. You paste the numbers your plate reader, cytometer or notebook produced. You shape them: subtract a blank, convert units, normalise to a control. You test them. You draw the figure the journal will print. One file holds all of it.
It does the job GraphPad Prism does, and the job of the spreadsheet most people open before Prism. It runs on Linux, Windows and macOS, it is free, and the source is open under the MIT licence.
The name comes from the dioptra of Hero of Alexandria, a sighting instrument for measuring angles and levels. The aim is the same as a good instrument’s: read the quantity accurately, show the scale, and do not decorate.
What it is built on
Four ideas run through every chapter of this manual.
Raw data is never changed. An analysis reads your table; it never writes to it. A transform makes a new column or a new table and remembers how. Excluding an outlier strikes the value through; it does not delete it. Every cell can tell you where it came from: typed by hand, pasted at a certain time, imported from a file and column, or computed by a formula.
Every assumption is shown. A t test assumes something about your data. So does an ANOVA, a regression, a chi-square test. Dioptra checks what can be checked, shows the check above the result, and names the alternative when a check fails. The defaults follow current statistical practice (Welch’s t test rather than Student’s, two-sided tests, Tukey–Kramer for all pairs, Holm rather than plain Bonferroni) and the results sheet says which one ran.
The file is the record. A .dioptra file is a zip of readable JSON. It holds the data,
each formula as text, each analysis as a specification, each graph as a specification, and
the history of commands. Open it in a year and every number comes back the same way.
The figure on screen is the figure you export. The plot engine writes one SVG; the PDF and PNG are made from it, with the fonts bundled, so a figure rendered on Linux is byte-identical on a Mac. Sizes are in millimetres and points, because that is how journals specify them.
How this manual is organised
- Getting started takes you from download to an exported figure in about ten minutes.
- Data tables explains the eight table kinds and how to choose one.
- Entering and shaping data covers paste, missing and excluded values, provenance and transforms. Formulas is the formula reference.
- Analyses explains the guided flow, assumption checks and p-value styles, then documents each analysis.
- Graphs explains the editor, themes, palettes and export, then each graph family.
- File format, Correctness and Performance are the record: what is stored, how the numbers were verified, and how fast it runs.
- Architecture and internals is for people who want to change the code.
- The complaint map lists what scientists dislike about the Prism workflow and what Dioptra does about each item.
The statistics are explained where they are used, briefly, with a reason for each default. If a chapter tells you to do something, it should also tell you why.
Getting started
This chapter takes you from nothing to an exported figure: install, paste a small data set, describe it, test it, draw it, export it, save it. It takes about ten minutes.
Install
Every release is published on the releases page. Pick the file for your system. The names below are those of version 0.4.0; later versions change only the number.
| System | File | What it is |
|---|---|---|
| Linux, any distribution | Dioptra_0.4.0_amd64.AppImage | One self-contained file. Make it executable and run it. |
| Debian, Ubuntu, Mint | Dioptra_0.4.0_amd64.deb | A package; installs a menu entry and pulls in WebKitGTK. |
| Fedora, RHEL, openSUSE | Dioptra-0.4.0-1.x86_64.rpm | The same, for rpm-based systems. |
| Windows 10 and 11 | Dioptra_0.4.0_x64_en-US.msi | The usual Windows installer. |
| Windows, per-user | Dioptra_0.4.0_x64-setup.exe | Installs without administrator rights. |
| macOS, Apple silicon (M1 and later) | Dioptra_0.4.0_aarch64.dmg | Open, drag Dioptra to Applications. |
| macOS, Intel | Dioptra_0.4.0_x64.dmg | The same, for Intel Macs. |
Linux
The AppImage needs nothing installed:
curl -LO https://github.com/ZahakJ/dioptra/releases/latest/download/Dioptra_0.4.0_amd64.AppImage
chmod +x Dioptra_0.4.0_amd64.AppImage
./Dioptra_0.4.0_amd64.AppImage
If it refuses to start with a message about FUSE, your distribution does not ship
libfuse2 by default (Ubuntu 22.04 and later). Install it (sudo apt install libfuse2),
or use the .deb.
On Debian and Ubuntu:
curl -LO https://github.com/ZahakJ/dioptra/releases/latest/download/Dioptra_0.4.0_amd64.deb
sudo apt install ./Dioptra_0.4.0_amd64.deb
On Fedora and other rpm systems:
curl -LO https://github.com/ZahakJ/dioptra/releases/latest/download/Dioptra-0.4.0-1.x86_64.rpm
sudo dnf install ./Dioptra-0.4.0-1.x86_64.rpm
Windows
Download Dioptra_0.4.0_x64_en-US.msi and run it. Windows may show a SmartScreen notice
because the installer is not signed with a paid certificate; choose More info, then
Run anyway. Dioptra uses the WebView2 runtime that ships with Windows 10 and 11.
macOS
Download the .dmg for your processor (the Apple menu, About This Mac, says “Apple M…”
for Apple silicon or “Intel” otherwise), open it and drag Dioptra into Applications. The
app is ad-hoc signed but not notarized (there is no paid Apple certificate), so the first
launch says “Apple could not verify Dioptra”. Control-click the app, choose Open, then
Open again; on macOS 15 and later, open it once, then go to System Settings → Privacy &
Security and click Open Anyway. Or run once in Terminal:
xattr -cr /Applications/Dioptra.app
macOS remembers the choice.
Paste your first data
Start Dioptra. You see an empty project: the project rail on the left (tables, analyses, graphs, layouts), the work area in the middle and the inspector on the right. The work area offers three ways in: paste data, open the bundled sample project (a cytokine study with a dose comparison, a dose-response fit and a survival curve, each with its analysis and figure), or open this manual.
Copy these three columns (from this page, a spreadsheet or a text file; tabs, commas and semicolons all work):
Control Low dose High dose
12.1 15.3 21.0
10.4 16.8 19.2
11.7 14.1 24.6
13.2 17.5 22.3
9.8 15.9 20.8
12.5 16.2 23.1
Choose File → Paste as new table (Ctrl-Shift-V; ⌘-Shift-V on a Mac). The dialog shows a preview, what it detected (separator, decimal mark, whether the first row is a header) and a suggested table kind with a small sketch of each kind. For three groups of replicate values the right kind is Column: each column is a group. Pick it, give the table a title such as “Plasma IL-6”, and press Create table.
Click any cell. The inspector says where it came from (“Pasted from the clipboard at …”). The status line at the bottom shows n, mean, SD and sum of the selection.
Describe it
Press Analyze (Ctrl-Shift-A). The list shows every analysis; those that do not fit a Column table are greyed with the reason. Choose Descriptive statistics and run it. The results sheet opens beside the table: one sentence per group in APA style, then the table of N, mean, SD, SEM, confidence interval, median and quartiles, then a short list headed Before you report this.
Test it
Press Analyze again and choose One-way ANOVA. The options panel holds the few choices that matter, each with one line on what it means: which groups, whether to use Welch’s ANOVA (which does not assume equal SDs), and which multiple comparisons to run (Tukey–Kramer for all pairs by default). Run it.
The sheet starts with the assumption checks: normality of residuals and equality of variances, each with its test result and what it means for this analysis. If one is in doubt it is open and amber, and it names the alternative (Welch’s ANOVA, or Kruskal–Wallis). Then the summary sentence, with the effect size and its confidence interval before the p value. Then the ANOVA table and the comparisons.
Change a number in the table. The sheet marks itself stale and recomputes when you look at it.
Draw it
On the results sheet press Make graph. Dioptra draws a column graph of the three groups with every data point shown, error bars, and a bracket for each comparison from the ANOVA, labelled in the project’s p-value style.
The graph is a live object. Click the Y axis title and the inspector shows its properties; type “IL-6 (pg/mL)”. Change the family to Box or Violin in the toolbar; the data and brackets follow. Choose a theme (Instrument, Night for slides, Journal for black-and-white print) and a palette.
Export it
Press Export (Ctrl-Alt-E). Choose PDF, SVG or PNG; set the size in millimetres or inches (85 × 70 mm is a single journal column); for PNG choose the resolution. The dialog states exactly what will be written, for example “1004 × 827 pixels at 300 dpi, 85 × 70 mm”. PDF embeds the fonts; SVG keeps text as text.
Save it
File → Save (Ctrl-S) writes a .dioptra file: the table, the formulas, both analyses,
the graph and the history of what you did. Close Dioptra and open the file again; every
result is recomputed from the data and comes out the same.
That is the whole loop. The rest of the manual explains each step in depth.
Data tables
A table in Dioptra is not a blank sheet. It has a kind, and the kind says what the rows and columns mean. That meaning is what lets Dioptra offer the right analyses and draw the right graph without asking you to explain your experiment twice.
Why not one flat sheet for everything? Because the same six numbers mean different things in different experiments. Six values in two columns might be two independent groups of three (an unpaired comparison) or three subjects measured twice (a paired one). The numbers cannot tell you which; the layout can. Prism made this choice, and it is the part of Prism people value most once they have learned it (the guided analysis, K1 in the complaint map). It is also the part they find hardest to learn (P9). Dioptra keeps the idea and tries to make the choice easier: each kind is explained in one sentence with a sketch when you create or paste a table, the paste dialog suggests a kind from the data, and when in doubt the Multiple variables kind is a safe default that can be reshaped into any other.
Shared vocabulary
- A data set is a named group of columns: one group in a Column table, one Y series in an XY table, one variable in a Multiple variables table. Data sets are lettered A, B, C, as in formulas.
- A data set has a subcolumn format. With replicate values each subcolumn holds one
replicate (
values(3)is three replicates side by side). With a summary format you type statistics computed elsewhere: mean, SD and N; mean, SEM and N; mean and a 95% confidence interval with N; mean and SD; mean and SEM; median and quartiles; lower and upper limits; or a single value. Summary formats exist because published papers and instrument software often give you only the summary, and with N present most analyses can still be computed exactly. - Row titles label rows: subjects, samples, levels of a factor.
- An empty cell is missing, never zero. Zero is a number.
The eight kinds
Column
Groups side by side, one column each.
| Control | Low dose | High dose |
|---|---|---|
| 12.1 | 15.3 | 21.0 |
| 10.4 | 16.8 | 19.2 |
| 11.7 | 14.1 | 24.6 |
Use it for one factor: treatments, genotypes, time points measured in separate subjects. Rows mean nothing unless the design is paired or repeated, in which case each row is one subject and you choose a paired analysis.
Analyses: descriptive statistics, t tests (one-sample, unpaired Welch or Student, paired, ratio paired), Mann–Whitney, Wilcoxon, one-way ANOVA (with Welch and Brown–Forsythe variants), repeated-measures one-way ANOVA, Kruskal–Wallis, Friedman, normality tests, outliers, correlation, ROC curves, Bland–Altman. Default graph: column (bar) with every point shown. Prism: Column table.
XY
An X column and one or more Y data sets.
| Dose (nM) | Control: rep 1 | rep 2 | Treated: rep 1 | rep 2 |
|---|---|---|---|---|
| 0.1 | 2.3 | 2.1 | 2.5 | 2.4 |
| 1 | 4.0 | 3.8 | 6.1 | 5.7 |
| 10 | 7.2 | 6.8 | 9.4 | 9.9 |
Use it whenever each value has an X coordinate that is a number: dose–response, time courses, standard curves, calibration. Each Y data set can have replicate subcolumns or a summary format (mean, SD, N).
Analyses: simple linear regression (with confidence and prediction bands and residual diagnostics), nonlinear regression (the curve-fitting library: 4PL and 5PL, exponential decay and association, Michaelis–Menten, binding, Gaussian, polynomials, custom equations), correlation, descriptive statistics, normality, outliers. Default graph: scatter. Prism: XY table.
Grouped
Rows are levels of one factor, data sets are levels of another, subcolumns are replicates.
| Male: 1 | Male: 2 | Male: 3 | Female: 1 | Female: 2 | Female: 3 | |
|---|---|---|---|---|---|---|
| Wild type | 12 | 14 | 11 | 14 | 15 | 13 |
| Knock-out | 9 | 8 | 10 | 8 | 7 | 9 |
Use it for two factors at once: genotype by sex, treatment by time. The question it answers is whether each factor matters and whether they interact.
Analyses: two-way ANOVA (type III sums of squares by default, with type I and II available; repeated measures on rows or columns), descriptive statistics, normality, outliers. Default graph: grouped column. Prism: Grouped table.
Contingency
Counts of subjects in each combination of two categories.
| Cured | Not cured | |
|---|---|---|
| Drug | 30 | 10 |
| Placebo | 18 | 22 |
Use it when each subject falls into one row category and one column category and you counted them. The cells hold counts, not percentages and not measurements.
Analyses: chi-square (with Yates’ correction for 2 × 2 as an option), Fisher’s exact test (2 × 2 and larger, by the network algorithm), with relative risk and odds ratio and their confidence intervals. Dioptra does not rewrite what you paste: a value such as 1.5 is kept, and the analysis tells you that it expected whole counts (Prism silently turns “1.23” into “123”, P7). Default graph: column. Prism: Contingency table.
Survival
Time in X; for each group, 1 for an event and 0 for a censored observation.
| Days | Group A | Group B |
|---|---|---|
| 12 | 1 | |
| 30 | 0 | |
| 41 | 1 |
Use it for time-to-event data: one row per subject, the time in X, and in that subject’s group column a 1 if the event happened at that time or a 0 if follow-up ended without it.
Analyses: Kaplan–Meier curves with Greenwood confidence intervals, median survival, log-rank and Gehan–Breslow–Wilcoxon comparisons, hazard ratios. Default graph: Kaplan–Meier with an at-risk table. Prism: Survival table.
Parts of whole
One data set of categories that add up to a whole.
| Count | |
|---|---|
| Type I | 42 |
| Type II | 17 |
| Type III | 9 |
Use it for fractions of a total: cell types in a population, reads per category.
Analyses: chi-square goodness of fit against equal proportions. Default graph: column (pie and donut are available, reluctantly: lengths are easier to compare than angles). Prism: Parts of whole table.
Multiple variables
One column per variable, one row per case, like a data frame.
| Age | Sex | BMI | Group |
|---|---|---|---|
| 54 | F | 23.1 | control |
| 61 | M | 27.4 | drug |
Use it for anything shaped like a spreadsheet of observations, and whenever you are not
sure. Each variable has a declared type: number, category (text labels such as
“control”), text (identifiers and notes, never analysed) or date. When you paste, the
type is inferred: all numbers become a number; ISO dates (2026-10-11) a date; text with
at most twelve distinct values a category; anything else text. Text stays text: a gene
called MARCH1 or SEPT2 is never turned into a date.
Analyses: descriptive statistics, t tests, Mann–Whitney and Wilcoxon between numeric variables, correlation (Pearson, Spearman, Kendall), principal component analysis, ROC, Bland–Altman, normality, outliers. To compare groups named in a category column, pivot long to wide: one data set per group (see Entering and shaping data). Default graph: column. Prism: Multiple variables table.
Nested
Groups as data sets, subgroups (such as animals) as subcolumns, observations stacked within each.
| Control: mouse 1 | mouse 2 | Drug: mouse 1 | mouse 2 |
|---|---|---|---|
| 3.2 | 2.9 | 4.4 | 4.1 |
| 3.5 | 3.1 | 4.0 | 4.3 |
| 3.0 | 3.3 | 4.2 | 3.9 |
Use it when you measured several things (cells, fields, wells) within each experimental unit (animal, culture, patient). This is the table kind that protects you from the most common error in cell biology: treating 300 cells from three mice as 300 independent observations. The experimental unit here is the mouse, so n = 2 per group, not 6.
Analyses: descriptive statistics per subcolumn, normality, outliers, and t tests and one-way ANOVA on the subgroup means, which the results sheet says it did. Mixed-effects models are planned (see the roadmap). Default graph: SuperPlot: every observation small and pale, each unit’s mean large, coloured by unit. Prism: Nested table.
Choosing quickly
Ask three questions.
- Do the values have a numeric X? Yes: XY (or Survival if X is time to an event).
- Are they counts of subjects in categories? Yes: Contingency (two classifications) or Parts of whole (one).
- How many grouping factors? One: Column. Two: Grouped. One, with several measurements inside each unit: Nested.
If none of these fits, or the data arrive in long format from another program, use Multiple variables and reshape later. Changing your mind costs nothing: transforms produce new tables and leave the original as it was.
Entering and shaping data
Most errors in a published figure happen before any statistics: a column pasted one row off, a gene name turned into a date, a value rounded by a spreadsheet’s display format, a baseline subtracted by hand in one column and forgotten in the next. This chapter is about how Dioptra takes data in and changes it, and how it keeps a record of both.
Typing
Select a cell and type. A number is stored as a number. Anything else (<LOD, n.d.,
see notes) is stored as text, shown in the cell, and never used in arithmetic. It is
not converted to zero and it is not thrown away. A text cell in a numeric column is
something you should see, so Dioptra shows it.
Start with = and you type a formula; see Formulas.
The grid behaves like a spreadsheet: arrows, Tab and Enter move; Shift extends the selection; F2 or Enter edits; Delete clears; Ctrl-D fills down; Ctrl-Z undoes. The full list is in Keyboard shortcuts.
Pasting
Paste (Ctrl-V into a table, or File → Paste as new table, Ctrl-Shift-V) accepts any delimited text: a block copied from Excel, LibreOffice or Google Sheets, a plate reader’s export, a CSV file opened in a text editor. Dioptra decides three things and tells you what it decided.
The separator. A tab anywhere means tab-separated (this is what spreadsheets put on the clipboard). Otherwise a semicolon or a comma that appears the same number of times on every line wins, semicolon first; then columns of whitespace-separated numbers; then commas.
The decimal mark. Much of the world writes one and a half as 1,5. With tab or
semicolon separators, Dioptra counts the fields that look like 1,5 and those that look
like 1.5; if comma decimals are the majority, commas are decimal marks, and the paste
report says so: “Commas were read as decimal marks (24 values looked like 1,5).” With
comma separators the decimal mark is always the point, because a comma cannot be both.
Some values are ambiguous: 1,234 is one thousand two hundred thirty-four in
Boston and one point two three four in Berlin. Dioptra does not guess. It keeps such values
as text, counts them in the report, and asks you to choose the decimal mark in the dialog.
The header. If the first row is text over columns of numbers, it is read as a header and becomes the data set names.
Missing-value codes are recognised and stored as empty: an empty field, -, ., NA,
N/A, NaN, #N/A, null. Everything else that is not a number stays as text.
What Dioptra never does on paste: turn SEPT2 into 2 September, read (1.5) as a
positive number, drop a minus sign, or take the rounded value a spreadsheet was displaying instead of the one it stored. When you paste from a
spreadsheet, check that the spreadsheet shows enough decimals: the clipboard carries what
is displayed. Importing the .xlsx file directly avoids this and reads the stored values.
Importing files
File → Import (Ctrl-I) reads:
- CSV, TSV and plain text, with the same detection as paste;
- Excel workbooks (
.xlsx,.xls) and OpenDocument spreadsheets (.ods), by a reader built into Dioptra, so it works the same on every system and needs no Office installation. Cell values are read as stored, at full precision, not as displayed. Date cells become ISO dates (2026-10-11) and the import notes say how many; - Prism
.pzfxfiles and Prism 10.prismfiles: the data tables come across, with their titles, kinds (nested and multiple-variables included), column names, subcolumn titles, row titles and excluded values. Summary formats Dioptra lacks are converted and the import notes say how: mean, %CV, N becomes mean, SD, N; mean with +/− error or with upper and lower limits becomes a “(mean)” data set plus a lower, upper data set. Date X values keep Prism’s elapsed time as X and the dates as row titles. An X error subcolumn is not imported (Dioptra tables have no X error). Prism’s analyses and graphs are not converted; run the analyses again in Dioptra, which takes a minute and gives you assumption checks you did not have before.
Each imported column remembers the file, sheet and column it came from.
Plate readers
Exports of four plate-reader programs are recognised by their content and read as plates:
| Program | File | Reads |
|---|---|---|
| Molecular Devices SoftMax Pro | .txt (UTF-16 or UTF-8), plate or time format | endpoint, kinetic, several wavelengths and plates |
| BMG Labtech MARS | .csv, .txt | endpoint plate grids |
| Agilent BioTek Gen5 | .txt | endpoint result grids (one read per label), kinetic time courses |
| Tecan Magellan | .xlsx with a Well positions column | endpoint labels, kinetic cycles |
Without a plate map, an endpoint file becomes a grouped table: rows A, B, … down, one data set per read with the plate columns 1, 2, … as subcolumns. A kinetic file becomes an XY table with time in seconds as X and one data set per well.
After choosing a plate file, Import asks for an optional plate map: paste a grid as
copied from a spreadsheet (row letters and column numbers optional), or one well, label
line per well. A label is a group (Control), a dose (10), or both (Drug A @ 10); in
the list form the dose may be a third column. Empty cells leave a well out.
1 2 3
A Blank Drug A @ 0.1 Drug A @ 1
B Blank Drug A @ 0.1 Drug A @ 1
With groups only, the plate becomes a column table, one data set per group. With doses, it becomes an XY table: X is the dose, one data set per group, and wells with the same group and dose are replicates. Kinetic reads keep time as X, with one data set per group.
Not read: spectrum and well-scan reads, SoftMax Pro group (calculation) blocks, Gen5
calculated results in kinetic files and MARS table-view exports. The import notes list what
was left out and any well values that were not numbers (OVRFLW, >4.000).
Replacing a table’s data
File → Replace table data… reads a file into the open table instead of a new one. The table keeps its name, data set names and column formulas; its analyses, graphs and layouts recompute. Data columns are filled left to right, skipping formula columns. A file with more columns adds data sets; a file with fewer clears the columns it does not reach. The file must give the same kind of table. One undo restores the old data.
This re-runs a project on new data by hand. Batch analysis does it for many files at once.
Missing values
An empty cell is missing. It is not zero, and zero is not missing. Analyses say how they
handle missing values: most use every available value in each group; paired and
repeated-measures analyses drop a subject (row) with any missing value and say how many
were dropped. Formulas propagate missing (A + 1 is missing where A is) while summaries
skip it (mean(A) is the mean of the values present).
Excluded values
Sometimes a value is real but should not be analysed: a well that dried out, a failed injection. Select it and press Ctrl-E (Edit → Exclude or include). The value stays in the table, struck through, and every analysis, formula and graph treats it as missing. Press Ctrl-E again to bring it back.
Why not delete it? Because the record matters. A deleted value is invisible to a reviewer, to your colleague, and to you in six months. An excluded value is still there, and the results sheet counts how many excluded values it left out. If you exclude, write down why; outlier tests (Grubbs, ROUT) can tell you which values are unusual, but whether to exclude one is a decision about the experiment, not about the numbers.
Provenance: where every cell came from
Click a cell. The inspector’s Cell panel says where its value came from:
- Typed by hand;
- Pasted from the clipboard at a time;
- Imported from a file, sheet and column;
- Computed by a formula, which it shows;
- Derived from another table by a transform.
If you edited a pasted or imported value by hand, the edit is recorded on top of the source, with the old value and the time. This is the provenance chain, and it is saved in the file. It answers the question every reviewer eventually asks: where did this number come from?
Transforms: new tables from old
Some operations change the shape of the data, not just its values. Dioptra does these as derived tables: you choose a transform from the Data menu, and a new table appears that remembers its source and the recipe. Change the source and the derived table follows. The original is never modified.
| Transform | What it does | Typical use |
|---|---|---|
| Transpose | Rows become data sets and data sets become rows | A plate reader that exports samples as rows |
| Pivot long → wide | One row per id, one data set per key | Tidy data from R or a database into a Column or Grouped layout |
| Pivot wide → long | A Multiple variables table of (id, variable, value) | Going the other way |
| Stack | Columns into one (group, value) table | Preparing for a by-group analysis elsewhere |
| Filter rows | Keep rows where a formula is true, e.g. A > 0 && !isna(B) | Dropping blanks or a subset |
| Sort | Order rows by a column | Reading the extremes |
| Select rows, select columns | Keep a range or a subset | Analysing part of a big table |
| Remove baseline | Subtract, divide or take percent of a baseline: the first row, the mean of chosen rows, another column, or a titled row | Blank subtraction; percent of time zero |
| Fraction of total | Each value as a fraction of its row, column or table total | Proportions |
| Normalize | Map one value to 0 and another to 1 (or 100%): min and max, first and last row, or numbers you give | Percent of maximal response |
| Average replicates | Replicate subcolumns become mean, SD and N | Summarising before plotting |
| Prune | Keep one row in k | Thinning a long time series for display |
For per-row arithmetic (log transforms, unit conversions, ratios) use a column formula instead: it is shorter, and it shows its recipe in the formula bar.
Exporting tables
File → Export table (Ctrl-Shift-E) writes CSV, TSV or an Excel workbook, at full precision. That is how you hand data to a colleague who uses something else; you never need it to move data between steps in Dioptra.
Formulas
A formula computes numbers from other numbers in the same table. You type it after an =,
either for a whole column (the formula bar above a column header) or for a single cell.
Two places a formula can live
A column formula fills every row of a column. = log10(A) in column B means “for each
row, B is the base-10 log of A in that row”. One formula, a million rows, no dragging.
A cell formula fills one cell, the way a spreadsheet does. Type =A3 * 2 into cell B3.
Fill-down (Ctrl-D) copies it to the rows below and moves the row numbers along: B4 gets
=A4 * 2. A column holds either a column formula or cell formulas, not both.
Computed cells show a small corner mark. Change the data and every formula that depends on
it is recomputed the next time you look. A formula that would depend on itself, directly or
through other columns, is refused when you type it, with the loop spelled out:
Circular reference: C → D → C.
Naming columns
| You write | It means |
|---|---|
A, B, … Z, AA | data set A, B, … (its first subcolumn) |
A.2 | data set A, subcolumn 2 (replicate 2, or the SD in a mean/SD/N data set) |
X | the X column of an XY or survival table |
`Dose (nM)` | the data set (or X column) with that header, in backticks |
`Dose (nM)`.3 | its third subcolumn |
A3 | one cell: data set A, row 3 (rows count from 1) |
A[3] | the same cell |
A[2:5] | rows 2 to 5 of A, both ends included |
A[:5], A[2:] | from the first row to row 5; from row 2 to the end |
[1, 2, 3] | a list of numbers you type yourself |
The letter X always means the X column, so the twenty-fourth data set is reached by its
name in backticks.
Values
A formula works with numbers, text ("mg/dL"), true/false, and missing. An empty cell
is missing. An excluded cell is missing too: it stays in the table, struck through, but
no formula or analysis uses it. NA (or na()) writes missing on purpose.
Missing spreads: A + 1 is missing wherever A is missing. Summaries such as mean skip
missing values instead, because that is almost always what you want from “the mean of A”.
A problem in one row does not stop the column. sqrt(A) on a row where A is −4 shows an
error in that cell (hover to read “sqrt: negative number”) and computes every other row.
Text in a numeric column (“<LOD”) is kept and shown; arithmetic on it gives an error in
that row, never a silent zero.
Arithmetic and logic
From tightest to loosest binding:
| Operators | Meaning | Example |
|---|---|---|
A[…] | pick rows | A[1:3] |
^ | power (right to left: 2^3^2 is 2^9) | B ^ 2 |
-x, +x | sign (-2^2 is −4) | -A |
*, /, % | multiply, divide, remainder (floored, like MOD) | A % 2 |
+, - | add, subtract | A - B |
== (or =), != (or <>), <, <=, >, >= | compare; gives true/false | A > 0 |
not x, !x | not | not isna(A) |
&&, and | both | A > 0 && B > 0 |
||, or | either | A < 0 or B < 0 |
True counts as 1 and false as 0 when stored in a column, which is how you make an event
code: = if(A > 30, 1, 0). Logic with a missing value follows common sense:
false && NA is false, true || NA is true, true && NA is missing.
A single number combines with a whole column row by row: A * 2, A - mean(A). Two
columns combine row by row. A list of a different length than the table cannot combine
with a column; that is an error, not a guess.
Row-by-row functions
| Function | What it gives | Example |
|---|---|---|
abs(x) | absolute value | abs(A - B) |
sqrt(x) | square root (error below 0) | sqrt(A) |
exp(x) | e to the power x | exp(-0.693 * X) |
ln(x) | natural log (error at or below 0) | ln(A) |
log(x) | natural log, same as ln (not base 10, unlike Excel) | log(A) |
log(x, b) | log in base b | log(A, 2) |
log10(x) | base-10 log | log10(A) |
log2(x) | base-2 log | log2(B / A) |
pow(x, y) | x to the power y | pow(10, A) |
sin, cos, tan | trigonometry, radians | sin(X * pi() / 180) |
asin, acos, atan | inverse trigonometry, radians | atan(A) |
sinh, cosh, tanh | hyperbolic functions | tanh(A) |
floor(x), ceil(x) | round down, round up | floor(A) |
round(x), round(x, n) | round to n decimals, halves away from zero (2.675 → 2.68) | round(A, 2) |
trunc(x) | drop the fraction | trunc(A) |
sign(x) | −1, 0 or 1 | sign(B - A) |
if(c, a, b) | a where c is true, b where false, missing where c is missing | if(A > 0, A, NA) |
isna(x) | true where x is missing (or excluded) | if(isna(A), 0, A) |
na() | a missing value | if(A < 0, na(), A) |
coalesce(a, b, …) | the first argument that is not missing | coalesce(A, B, 0) |
clamp(x, lo, hi) | x limited to the range lo to hi | clamp(A, 0, 100) |
pct(x, ref) | 100 × x / ref | pct(A, A1) |
normalize(x, lo, hi) | maps lo to 0 and hi to 1 | normalize(A, A1, max(A)) * 100 |
convert(x, "from", "to") | unit conversion (see below) | convert(A, "nM", "µM") |
convert(x, "from", "to", "substance") | mass ↔ molar conversion | convert(A, "mg/dL", "mmol/L", "glucose") |
row() | the row number, from 1 | if(row() <= 3, NA, A) |
pi() | π | 2 * pi() * X |
Summaries (one number from many)
These skip missing, excluded and text cells. Give them a column, a slice, a list, or
several of these at once: max(A, B) is the largest value in A and B together. (For a
row-by-row limit use clamp, not min/max.)
| Function | What it gives | Example |
|---|---|---|
mean(x) | arithmetic mean | A - mean(A[1:3]) |
median(x) | middle value | A / median(A) |
sd(x) | sample standard deviation (n − 1) | (A - mean(A)) / sd(A) |
sem(x) | standard error of the mean, sd/√n | sem(A) |
var(x) | sample variance (n − 1) | var(A) |
sum(x) | total | A / sum(A) |
count(x) | how many numbers | count(A) |
min(x), max(x) | smallest, largest | (A - min(A)) / (max(A) - min(A)) |
range(x) | max − min | range(A) |
quantile(x, q) | the q-quantile, q from 0 to 1, by linear interpolation (Excel’s PERCENTILE.INC) | quantile(A, 0.9) |
geomean(x) | geometric mean (all values must be positive) | A / geomean(A) |
cv(x) | coefficient of variation, in percent: 100 × sd / mean | cv(A) |
first(x), last(x) | first and last non-missing value | A - first(A) |
The descriptive-statistics analysis computes quartiles by the (n + 1)p rule, as Prism
does; quantile uses the spreadsheet rule. For small samples the two can differ slightly.
Series (whole-column operations that give a column back)
| Function | What it gives | Example |
|---|---|---|
lag(x), lag(x, k) | the value k rows above (missing for the first k rows) | A - lag(A) |
lead(x), lead(x, k) | the value k rows below | lead(A) - A |
diff(x) | x minus the row above | diff(X) |
cumsum(x) | running total; a missing row stays missing and is skipped in the total | cumsum(A) |
rank(x) | rank from smallest (1), ties share the average rank | rank(A) |
zscore(x) | (x − mean) / sd | zscore(A) |
normalize(x) | (x − min) / (max − min) | normalize(A) * 100 |
baseline(x, r1, r2, …) | x minus the mean of x at the listed rows | baseline(A, 1, 2, 3) |
seq(a, b), seq(a, b, step) | the list a, a + step, …, b | baseline(A, seq(1, 5)) |
Units
convert(x, "from", "to") knows these units. Metric prefixes work on every base unit:
k (kilo), d (deci), c (centi), m (milli), µ or u or mc (micro), n (nano), p (pico),
f (femto).
| Quantity | Units |
|---|---|
| mass | g, kg, mg, µg, ng, pg |
| volume | L (or l), dL, mL, µL, nL, cc |
| amount | mol, mmol, µmol, nmol, pmol |
| concentration | M, mM, µM, nM, pM; or any mass or amount over a volume: mg/dL, g/L, mmol/L, µg/mL |
| time | s, ms, µs, min, h, d, wk |
| length | m, km, cm, mm, µm, nm |
| pressure | Pa, kPa, hPa, mmHg, atm, bar |
| temperature | C, F, K (or °C, °F) |
Mass and molar concentrations convert through a substance’s molar mass, so you must name the substance; Dioptra will not guess:
| Substance | Conversion |
|---|---|
"glucose" | mg/dL ÷ 18.016 = mmol/L |
"cholesterol" | mg/dL ÷ 38.67 = mmol/L |
"triglycerides" | mg/dL ÷ 88.57 = mmol/L |
"creatinine" | mg/dL × 88.42 = µmol/L |
"urea" | molar mass 60.06 g/mol |
Where formulas are also used
Filter rows takes a formula that is true for the rows to keep: A > 0 && !isna(B).
Differences from a spreadsheet
log(x)is the natural log. Uselog10(x)for base 10.- Slices are inclusive and count from 1:
A[1:3]is three rows. %is the remainder, not “per cent”.- There are no
$absolute references; a column formula already means “this column, every row”, and fill-down moves only single-cell references such asA3. - Function names are case-insensitive (
MEAN(A)works); column letters are capitals.
Statistical analyses: t tests, ANOVA, regression and more
How to run a statistical test in Dioptra: select the data, press Analyze (Ctrl-Shift-A), choose the test and its options, and read the results sheet, which shows the assumption checks first and the effect size with its confidence interval.
An analysis in Dioptra is an object in your project, like a table or a graph. It holds a specification (which table, which test, which options) and a results sheet computed from it. Change the data and the sheet recomputes. Change an option and the sheet recomputes. Delete the analysis and the history still records that it existed.
This page explains the flow, the parts of a results sheet, and the defaults. The analysis reference documents each analysis: when to use it, what it assumes, its options and outputs, its reference, and a worked example.
The flow
- Select data and press Analyze (Ctrl-Shift-A, or the Analyze… button above the grid). The list holds all twenty analyses. Those that fit the table kind are active; the others are shown greyed with the reason (“needs a contingency table”, “needs at least three data sets”). You see what exists even when it does not apply.
- Choose options. One compact panel with the few choices that matter, each with one line saying what it means. The defaults are the ones a careful statistician would choose without knowing more about your experiment.
- Run. The results sheet opens beside the table.
- Make a graph from the sheet with one click. Brackets for comparisons and fitted curves come from the analysis and stay linked to it.
- Change anything. The sheet and the graph mark themselves stale and recompute when you look at them.
What a results sheet contains
A results sheet is structured, not a block of text. From top to bottom:
Assumption checks. What the test rests on, what was checked, and what the check means for this result. Collapsed to one line when everything passed; open when something did not.
Summary sentences. The result in words, in the style of a careful methods section:
Welch’s t test: the means differ by 3.2 units (95% CI 1.1 to 5.3), t(17.4) = 3.21, p = .005.
The estimate and its confidence interval come first. The p value comes last, as one piece of evidence among several.
A methods sentence you can paste into a paper: which test, which variant, two-sided or not, which correction for multiple comparisons, how many values were used, and the Dioptra version.
Tables. Every number at full precision, displayed to a sensible number of digits. Copy a single number, a table, or the whole sheet as tab-separated text or Markdown (Copy sheet).
Effect sizes with confidence intervals. Mean differences, Cohen’s d and Hedges’ g, eta squared, odds ratios and relative risks, correlation coefficients, hazard ratios, AUC. In the base product, always shown. An effect size says how big the difference is; a p value only says whether the data are surprising if there were none.
A checklist headed Before you report this: the things the test cannot check for you. Are the values independent? Was the pairing decided before the experiment? Is the experimental unit the row, or something larger?
Assumption checks
Every parametric test assumes things about the data: that values are independent, that they come from a roughly Gaussian distribution, that the groups have similar spreads. Independence can only be judged from the design, so it goes in the checklist. The rest can be checked, and Dioptra checks it, at α = 0.05 for every check:
| Check | How | Status |
|---|---|---|
| Normality | Shapiro–Wilk (3 ≤ n ≤ 5000), with D’Agostino–Pearson as a second opinion from n = 8; applied to whatever the test assumes is Gaussian: each group, the paired differences, or the residuals | Pass when every p ≥ .05; caution when the smallest p is between .01 and .05 or only D’Agostino–Pearson objects; fail below .01. Not checked below n = 3 |
| Equal variances | Brown–Forsythe (Levene’s test on medians) decides; Bartlett’s test is shown beside it | When the test in use does not assume equal variances (Welch), it is shown for information only |
| Sample size | Count per group against the analysis’s minimum | Fail below 2 values; caution below the minimum |
| Outliers | Grubbs’ test at α = 0.05 on each group | A note only, never a failure: an unusual value is a question for the experiment |
| Values used | How many values were used, and what was left out (empty, text, excluded) | Caution when something was dropped |
A failed check does not stop the analysis. It says what the failure means here and names the alternative: Welch’s ANOVA when variances differ, a rank-based test when the distribution is far from Gaussian, more data when n is too small.
Two warnings about normality tests, which the sheet repeats where they matter. With small samples they have little power: a pass at n = 4 tells you almost nothing. With large samples they detect departures too small to matter, and the t test is robust to those. Look at the data as well. Use the checks as a prompt, not as an automatic switch; choosing the test after seeing which one gives the smaller p value is the p-hacking the analysis log is there to make visible.
The defaults, and why
Welch’s t test, not Student’s. Student’s t test assumes the two groups have equal variances. When they do, Welch’s test gives almost the same answer. When they do not, and especially when the smaller group has the larger variance, Student’s test can be badly wrong. Welch costs nothing when the assumption holds and protects you when it fails, so it is the default. Student’s test is one click away for anyone who needs it.
Two-sided. A one-sided test halves the p value in the direction you predicted and cannot detect an effect in the other. It is legitimate only when the direction was fixed before the data were seen and an effect the other way would be treated as no effect. That is rare, so two-sided is the default, and choosing one-sided adds a sentence to the methods line saying so.
Tukey–Kramer for all pairs. After an ANOVA the default compares every pair of groups with Tukey–Kramer, which handles unequal group sizes. For each group against a control, choose Dunnett; for a few pairs chosen in advance, Holm or Šidák. Each of these controls the chance of any false positive across the family of comparisons, with the most power available for that question. Plain Bonferroni is available but needlessly conservative; Holm is uniformly better. Benjamini–Hochberg controls the false discovery rate instead, for screens with many comparisons.
Type III sums of squares for two-way ANOVA, with sum-to-zero coding, so that unbalanced designs give the answer most textbooks and Prism give. (A frequent error is type III with dummy coding; Dioptra’s tests check that it does not make it.)
Exact tests when affordable. Mann–Whitney and Wilcoxon p values are exact when there are no ties and the samples are small enough to enumerate; Fisher’s test is exact for tables larger than 2 × 2 too; Spearman’s p is exact for n ≤ 9. The sheet says which method produced the p value.
P-value styles
A p value can be written several ways. The project’s style is set once (inspector, Project, P values) and applies everywhere: results sheets, summary sentences, and the labels on graph brackets. Changing it changes them all, and never changes the number underneath.
| Style | p = 0.0049 | p = 0.00004 | Typical use |
|---|---|---|---|
| APA (default) | p = .005 | p < .001 | Psychology, many biomedical journals |
| NEJM | P=0.005 | P<0.001 | Clinical journals |
| GP | P = 0.0049 (**) | P < 0.0001 (****) | Matching a Prism figure |
| Exact | p = 0.0049 | p < 0.0001 | Four significant digits |
The GP style uses GraphPad’s asterisks: * < .05, ** < .01, *** < .001, **** < .0001, decided on the stored value, not the rounded one. It is available because some journals and labs require it. It is not the default because asterisks turn a continuous measure of evidence into a grade, and the word “significant” invites the reader to stop at the grade.
Reproducibility
Each analysis is saved in the project file as its specification, so reopening the file
recomputes every result from the data. Each run, option change and deletion is a command
in the history (history.jsonl inside the file), so the record of what was tried is kept
with what was reported.
Analysis reference: every statistical test
One entry per analysis kind: when to use it, the tables it accepts, its options (JSON keys of the spec, with defaults; every key may be left out), what it checks, what the results sheet contains, the method it rests on, and a worked example. The general rules (how missing, excluded and summary values are handled, the assumption-check policy, the sentence format) are in METHODS.md, Results sheets.
Every sheet has: assumption checks (shown first), summary sentences, a methods line, typed
sections, and a checklist of what the analysis assumes and cannot tell you. columns are
flat column indices (the X column, when there is one, is 0); data sets are named by any of
their subcolumns. An empty columns means “the first data sets” (or all, as stated).
Multiple-comparison requests have the shape
{"method": {"name": "tukey" | "dunnett" (+ "control") | "sidak" | "holm" | "bonferroni" | "dunn" | "benjamini_hochberg"}, "comparisons": {"kind": "all_pairs" | "vs_control" (+ "control") | "selected" (+ "pairs")}}.
The default graph family is what CreateGraph picks for an analysis when family is empty
or "auto".
Descriptive statistics (descriptive)
How to get descriptive statistics in Dioptra (mean, SD, SEM, confidence interval of the mean, median, quartiles): select a table, press Analyze (Ctrl-Shift-A) and choose Descriptive statistics.
- When: describe each data set (n, mean, SD, SEM, CI of the mean, quartiles, range, CV, geometric mean).
- Tables: column, XY, grouped, variables, nested.
- Options:
columns[] (all),ci_level0.95,percentiles[] (extra, 0–100). - Checks: values used (text and excluded cells counted); independence and normality stated, not tested.
- Outputs: Descriptive statistics table. Graph:
column. - Method: descriptive; quartiles by the (n + 1)p rule.
- Example:
- Data: PlantGrowth
ctrl(n = 10). - Result: M = 5.032, SD = 0.5831, 95% CI [4.615, 5.449].
- Median 5.155, quartiles by Prism’s rule.
- Checks: every value used.
- Data: PlantGrowth
t test (unpaired, Welch, paired, one-sample) (t_test)
How to run a t test in Dioptra: enter the two groups as data sets of a column table (one row per subject for a paired test), press Analyze (Ctrl-Shift-A) and choose t test, then pick Welch, unpaired, paired, ratio paired or one-sample.
- When: compare two means (unpaired or paired), a ratio of paired values, or one mean with a hypothetical value.
- Tables: column, variables, nested.
- Options:
columns(first two data sets),variant"welch"(also"unpaired"= Student,"paired","ratio_paired","one_sample"),mu0,alternative"two_sided"("greater": the first data set is larger),ci_level0.95. - Checks: values used (pairs dropped for a missing side), sample size, normality of each group / the paired differences / the log ratios, equal variances (informational for Welch), outliers (Grubbs note), effective pairing (correlation of the pairs), independence. Unpaired and one-sample tests also run from mean/SD/N.
- Outputs: Descriptive, Test (t, df, two-sided P, one-sided P with its direction),
Difference (estimate, SE, CI), Effect size (Cohen’s d, Hedges’ g with approximate CIs,
R²), Normality, Equal variances, Ratio (ratio paired), Pairing. Brackets: the two groups.
Graph:
column(unpaired, one-sample),pairedpreset (paired): bars with a line per subject. - Method: ttest.
- Example:
- Data: the
sleepdata, drug1 and drug2 as columns, paired by row. - Paired: the mean of the differences was 1.6 (95% CI 0.7 to 2.5), t(9) = 4.06, p = .003, Hedges’ g = 1.17.
- Welch (as unpaired groups): 1.6 (95% CI -0.2 to 3.4), t(17.8) = 1.86, p = .079.
- The pairing removes the between-subject scatter: same difference, much narrower CI.
- Data: the
Mann–Whitney test (mann_whitney)
How to run a Mann–Whitney test in Dioptra: enter the two groups as data sets of a column table, press Analyze (Ctrl-Shift-A) and choose Mann–Whitney test.
- When: two unpaired groups, no assumption of Gaussian distributions.
- Tables: column, variables.
- Options:
columns(first two),alternative"two_sided",exactnull (automatic: exact without ties up to n₁n₂ = 10,000;trueforces exact,falseasymptotic),ci_level0.95. - Checks: values used, sample size, similar shapes (spread, by Brown–Forsythe), exact or approximate P, independence. Not possible from summary data.
- Outputs: Descriptive (medians, quartiles), Test (U, U₁, rank sums, z, P), Difference
(Hodges–Lehmann shift with Moses CI and achieved level), Effect size (rank-biserial r),
Equal variances. Graph:
boxwith a bracket. - Method: mannwhitney.
- Example:
- Data:
sleepdrug1 against drug2 as independent groups. - The Hodges–Lehmann shift was 1.4 (95% CI -0.1 to 3.6), U = 25.5, p = .069.
- Medians 1.75 and 0.35; ties, so P is the normal approximation.
- Check “Exact or approximate P” is a caution.
- Data:
Wilcoxon signed-rank test (wilcoxon)
How to run a Wilcoxon signed-rank test in Dioptra: enter the paired values as two data sets of a column table, one subject per row, press Analyze (Ctrl-Shift-A) and choose Wilcoxon signed-rank test.
- When: paired values (two columns) or one column against a hypothetical median, no Gaussian assumption.
- Tables: column, variables, nested.
- Options:
columns(one: againstmu; two: paired),mu0,alternative"two_sided",ci_level0.95. - Checks: values used, sample size (6 pairs), symmetric differences (skewness), exact or approximate P, independence.
- Outputs: Descriptive (medians of each column and of the differences), Test (W+, W−,
W, zeros dropped, z, P), Pseudo-median (Hodges–Lehmann with CI). Graph: the
pairedpreset (bars with connecting lines) (paired) orbox. - Method: wilcoxon.
- Example:
- Data:
sleep, paired by row. - The Hodges–Lehmann median of the differences was 1.3 (95% CI 0.8 to 2.8).
- W+ = 45, W− = 0, p = .008 (normal approximation: the differences have ties).
- Symmetry check passes.
- Data:
One-way ANOVA (one_way_anova)
How to run a one-way ANOVA in Dioptra: enter each group as a data set of a column table, press Analyze (Ctrl-Shift-A) and choose One-way ANOVA, and pick the multiple comparisons.
- When: three or more independent groups, compare means.
- Tables: column, nested.
- Options:
columns[] (all),post_hocTukey all pairs (nullfor none),welchfalse,brown_forsythefalse. - Checks: values used, sample size, normality of the residuals, equal variances (Brown–Forsythe; informational with Welch), outliers, independence. Runs from mean/SD/N (exact), with Bartlett for the variances.
- Outputs: Descriptive, ANOVA table (SS, df, MS, F, P), tests not assuming equal
variances (Welch, Brown–Forsythe), Effect size (η², ω², Cohen’s f), multiple comparisons
(Comparison, mean difference, CI, q or t, unadjusted and adjusted P, significant,
asterisks), Normality, Equal variances. Tukey for all pairs, Dunnett for a control,
Holm–Šidák for selected pairs, Šidák/Bonferroni/Benjamini–Hochberg on request. Brackets:
every comparison (up to six; else the significant ones). Graph:
column. - Method: anova, posthoc.
- Example:
- Data: PlantGrowth (ctrl, trt1, trt2; 10 each).
- F(2, 27) = 4.85, p = .016, η² = .26 (ω² = .20).
- Tukey: trt1 vs. trt2 differs (mean difference -0.87, 95% CI -1.56 to -0.17, p = .012).
- The graph carries three brackets: p = .391, p = .198, p = .012.
Repeated-measures ANOVA (repeated_measures_anova)
How to run a repeated-measures one-way ANOVA in Dioptra: enter one subject per row and one condition per data set of a column table, press Analyze (Ctrl-Shift-A) and choose Repeated-measures one-way ANOVA.
- When: each row is one subject measured under every condition (columns).
- Tables: column.
- Options:
columns[] (all),gg_correctiontrue,post_hocall pairs. - Checks: values used (incomplete rows dropped, counted), sample size, sphericity (Mauchly; pass when corrected), normality of the residuals, independence of subjects.
- Outputs: Descriptive, ANOVA table (treatment, subjects, residual, total), corrected
and uncorrected tests (none, Greenhouse–Geisser, Huynh–Feldt, lower bound, with ε),
Sphericity (Mauchly W, χ², df, P), Effect size (partial η²), paired multiple comparisons
(paired t tests with Holm–Šidák, or the adjustment asked; CIs for Bonferroni and Šidák),
Normality. Graph: the
pairedpreset (bars with connecting lines) with brackets. - Method: anova
rm_anova, posthocp_adjust. - Example:
- Data: 6 subjects × Pre, Mid, Post.
- Greenhouse–Geisser: F(1.53, 7.65) = 61.37, p < .001, partial η² = .92, ε = .77.
- Every pair differs after Holm–Šidák (largest adjusted p = .007).
- Sphericity passes (corrected P in use).
Two-way ANOVA (two_way_anova)
How to run a two-way ANOVA in Dioptra: put one factor in the rows and the other in the data sets of a grouped table, replicates in subcolumns, press Analyze (Ctrl-Shift-A) and choose Two-way ANOVA.
- When: two factors: rows of a grouped table (factor A) and its data sets (factor B), replicates in subcolumns.
- Tables: grouped.
- Options:
ss_type"iii"(also"i","ii"),post_hocnull (any request compares the cell means within each row on the pooled residual variance: Tukey for all pairs, else the adjustment named),repeatednull ("rows": each row’s subcolumn is one subject measured in every data set;"columns": each subcolumn of a data set is one subject measured in every row). - Checks: values used, replicates in every cell, normality of the residuals, equal variances across cells, independence; with a repeated factor: sphericity stated (not tested), balance.
- Outputs: ANOVA table (Source, SS, df, MS, F, P, partial η²) of the chosen type, Cell
means, within-row comparisons, Normality, Equal variances. The interaction sentence comes
first. With a repeated factor: the split-plot table (interaction, repeated factor,
between factor, subjects, residual). Graph:
column(grouped). - Method: anova
two_way_anova; the split-plot partition is the standard one for one between and one within factor (sequential with unequal groups). - Example:
- Data: 2 rows × 2 data sets × 3 replicates.
- Interaction F(1, 8) = 8.02, p = .022: the column effect depends on the row.
- Row factor F(1, 8) = 18.69, p = .003; column factor F(1, 8) = 72.20, p < .001.
- Tukey within rows: B exceeds A in both rows (p = .004 and p < .001).
Kruskal–Wallis test (kruskal_wallis)
How to run a Kruskal–Wallis test in Dioptra: enter each group as a data set of a column table, press Analyze (Ctrl-Shift-A) and choose Kruskal–Wallis test.
- When: three or more independent groups, no Gaussian assumption.
- Tables: column, nested.
- Options:
columns[] (all),dunntrue. - Checks: values used, chi-square approximation (groups of 5 or more), similar shapes stated, independence.
- Outputs: Descriptive (medians, quartiles, mean ranks), Test (H, df, P, ε², tie
correction), Dunn’s comparisons with Holm adjustment. Graph:
boxwith brackets. - Method: anova
kruskal_wallis, posthocdunn_kruskal. - Example:
- Data: PlantGrowth.
- H(2) = 7.99, p = .018, ε² = .28.
- Dunn (Holm): trt1 vs. trt2 differs (p = .015).
- The other two comparisons do not reach adjusted p < .05.
Friedman test (friedman)
How to run a Friedman test in Dioptra: enter one subject per row and one condition per data set of a column table, press Analyze (Ctrl-Shift-A) and choose Friedman test.
- When: matched rows (subjects) under three or more conditions, no Gaussian assumption.
- Tables: column.
- Options:
columns[] (all),dunntrue. - Checks: values used (incomplete rows dropped), chi-square approximation (10 subjects), independence.
- Outputs: Descriptive (medians, mean ranks), Test (χ², df, P, Kendall’s W, subjects
used and dropped), Dunn’s comparisons (Holm). Graph: the
pairedpreset (bars with connecting lines). - Method: anova
friedman, posthocdunn_friedman. - Example:
- Data: 6 subjects × Pre, Mid, Post.
- χ²(2, N = 6) = 12.00, p = .002, Kendall’s W = 1.00 (every subject ranks the conditions alike).
- Dunn (Holm): Pre vs. Post differs (p = .002).
- Chi-square approximation flagged (fewer than 10 subjects).
Chi-square test (chi_square)
How to run a chi-square test in Dioptra: enter the counts in a contingency table, press Analyze (Ctrl-Shift-A) and choose Chi-square test.
- When: association between the rows and columns of a table of counts; goodness of fit to equal proportions for a parts-of-whole table (or a one-column table).
- Tables: contingency, parts.
- Options:
yatesfalse (2 × 2 only). - Checks: counts are whole numbers (the bad cell is named), expected counts (Cochran’s rule, Fisher offered), independence of subjects.
- Outputs: Observed counts with totals, Expected counts, Test (Pearson χ², Yates, G, and
Fisher’s exact P for 2 × 2), Association (Cramér’s V, φ), Adjusted residuals, and for 2 × 2
the relative risk (Katz CI), odds ratio (Woolf CI) and risk difference (Wald CI).
Goodness of fit: observed and expected, χ², df, P. Graph:
column. - Method: contingency.
- Example:
- Data: 2 × 2 counts [[12, 5], [7, 15]] (rows = groups, columns = outcome).
- Risk 0.706 vs. 0.318: relative risk 2.2 (95% CI 1.1 to 4.4).
- χ²(1, N = 39) = 5.77, p = .016, Cramér’s V = .38; Fisher p = .025 shown alongside.
- Expected counts pass (smallest 8.28).
Fisher’s exact test (fisher_exact)
How to run Fisher’s exact test in Dioptra: enter the counts in a contingency table, press Analyze (Ctrl-Shift-A) and choose Fisher’s exact test.
- When: a 2 × 2 table, especially with small counts; r × c tables by the Fisher–Freeman–Halton test within its work limit.
- Tables: contingency (2 or more data sets).
- Options:
alternative"two_sided"("greater": odds ratio > 1; 2 × 2 only). - Checks: counts are whole numbers, fixed margins stated, independence.
- Outputs: Observed counts, Test (two-sided and one-sided P, sample and conditional MLE
odds ratios with the exact CI), Effect size (relative risk, odds ratio, risk difference).
r × c: exact P, probability of the observed table, number of tables, Cramér’s V; a clear
message when the table is too large. Graph:
column. - Method: contingency
fisher_exact_2x2,fisher_exact_rxc. - Example:
- Data: the lady tasting tea, [[3, 1], [1, 3]].
- Two-sided p = .486.
- Conditional MLE odds ratio 6.41 (95% CI 0.21 to 626).
- Relative risk 3.0 (95% CI 0.50 to 18.0).
Correlation (Pearson and Spearman) (correlation)
How to compute a correlation in Dioptra: enter the variables in an XY or Multiple variables table, press Analyze (Ctrl-Shift-A) and choose Correlation.
- When: association between two variables, or a matrix of several.
- Tables: XY (X with each data set by default), variables (numeric variables), column (data sets paired by row).
- Options:
columns[] (all),method"pearson"(also"spearman","kendall"),ci_level0.95. - Checks: values used (pairwise deletion counted), sample size, normality of each variable (Pearson), linear or monotonic relationship stated, outliers, independence.
- Outputs: Correlation table (pair, n, r, CI, r², P, Holm-adjusted P for several pairs),
the matrix for more than two variables, Normality. Graph:
scatterof the pair, orcorrelation_matrix. - Method: correlation.
- Example:
- Data: X = 1…10, Y = 2.1, 3.9, …, 19.7.
- r(8) = .999 (95% CI .998 to 1.000), p < .001, n = 10.
- Spearman on request: ρ with the Bonett–Wright CI.
- Linearity is stated, not tested: look at the scatter plot.
Linear regression (linear_regression)
How to run a simple linear regression in Dioptra: enter X and Y in an XY table, press Analyze (Ctrl-Shift-A) and choose Simple linear regression.
- When: a straight line through XY data, each data set separately.
- Tables: XY (replicates are separate points; mean/SD/N rows give their means).
- Options:
datasets[] (all),through_originfalse,bands"none"(also"confidence","prediction","both", for graphs),ci_level0.95,residualsfalse (adds a residuals table). - Checks: linearity (runs test), replicates test, normality of the residuals, equal scatter (|residual| against X), influential points (standardized residual, Cook’s distance), X without error stated, independence (Durbin–Watson shown).
- Outputs: Best-fit values (slope, intercepts with SE, CI, t, P), Goodness of fit (n,
R², Sy.x, SS, F, P, Durbin–Watson), Runs test, Replicates test, Normality, Residuals.
Graph:
scatterwith the line and its band (linear and log X). - Method: regression
simple_linear_regression. - Example:
- Data: X = 1…10, Y = 2.1, 3.9, …, 19.7.
- Slope 1.99 (95% CI 1.94 to 2.05), Y-intercept 0.060 (95% CI -0.280 to 0.400).
- R² = .999, F(1, 8) = 7052.15, p < .001.
- Runs test passes.
Nonlinear regression and curve fitting (dose-response, EC50) (nonlinear_regression)
How to fit a curve in Dioptra, such as a dose-response curve for an EC50: enter X and Y in an XY table, press Analyze (Ctrl-Shift-A) and choose Nonlinear regression (curve fit), and pick an equation.
- When: fit a model (dose-response, exponential, enzyme kinetics, binding, growth, or a formula) to XY data.
- Tables: XY.
- Options:
datasets[] (all, each fitted separately),equation{"name": "four_parameter_logistic"}(snake_case names such asone_phase_decay,michaelis_menten,log_inhibitor_variable_slope, Prism names and the aliases 4PL, 3PL, 5PL, line, MM; or{"name": "...", "formula": "Y = A*exp(-K*X) + C"}),weighting"none"(also"relative_y2"= 1/Y²,"poisson_y"= 1/Y,"inverse_sd2"= 1/SD² of the replicates),robustfalse,constraints[] ({"kind": "fixed" | "min" | "max", "param", "value"},{"kind": "shared", "param"}for a global fit),initial{} (by parameter name),compare_withnull (a second equation),profile_cifalse,rout_qnull (ROUT outlier removal at Q percent),ci_level0.95,bands"confidence"(for graphs). - Checks: values used, convergence (hit constraint, ambiguous, not converged), runs test, replicates test, normality of the residuals, outliers (ROUT when asked, otherwise stated), X as log(concentration) for the log equations, scatter of Y stated, independence.
- Outputs: Best-fit values (value, SE, CI, profile CI when asked, status, dependency),
Derived values (EC50 from LogEC50, half-life, Span…), Goodness of fit (n, df, R², adjusted
R², SS, Sy.x, AICc), Convergence, Runs test, Replicates test, Outliers (ROUT), Model
comparison (extra-sum-of-squares F test and AICc with probabilities and the preferred
model), Normality. Graph:
scatterwith each curve and its band (linear and log X). - Method: nonlinear, equations.
- Example:
- Data: 4PL dose-response, 11 log concentrations in duplicate.
- LogEC50 = -6.45 (95% CI -6.54 to -6.35), so EC50 = 3.58E-7 (95% CI 2.88E-7 to 4.45E-7).
- HillSlope = 0.75 (95% CI 0.63 to 0.87); R² = .995.
- Against the fixed-slope model: F(1, 18) = 14.52, p = .001; AICc prefers the variable slope (99.2%).
Normality tests (Shapiro–Wilk, D’Agostino) (normality)
How to test for normality in Dioptra: select a table, press Analyze (Ctrl-Shift-A) and choose Normality tests.
- When: look at the distribution of each data set.
- Tables: column, XY, grouped, variables, nested.
- Options:
columns[] (all),tests["shapiro_wilk", "dagostino_pearson"](also"anderson_darling","kolmogorov_smirnov"= Lilliefors). - Checks: values used (tests that cannot run are named), power of the tests (n < 8 or n > 500), what the tests mean.
- Outputs: Normality tests (data set, test, n, statistic, P, passed), Descriptive (mean,
SD, skewness, excess kurtosis). Graph:
qq. - Method: normality.
- Example:
- Data: PlantGrowth.
- ctrl: Shapiro–Wilk W = 0.96, p = .747; D’Agostino–Pearson K² = 0.23, p = .890.
- All three groups are consistent with a Gaussian distribution.
- That is absence of evidence, not proof.
Identify outliers (Grubbs, ROUT) (outliers)
How to find outliers in Dioptra: select a table, press Analyze (Ctrl-Shift-A) and choose Identify outliers.
- When: look for values that may be errors, one data set at a time.
- Tables: column, XY, grouped, variables, nested.
- Options:
columns[] (all),method{"method": "rout", "q": 1}(Q in percent) or{"method": "grubbs", "alpha": 0.05}. - Checks: values used, normality of the values without the flagged ones, decision to exclude (flagged values are never removed).
- Outputs: Flagged values (data set, row, cell, value), Before and after (n, flagged,
mean and SD with and without), Grubbs’ steps, Normality. Graph:
strip. - Method: outliers.
- Example:
- Data: 10.1, 9.8, 10.3, 10.0, 9.9, 10.2, 25, 10.1.
- ROUT (Q = 1%) flags 25 (row 7); so does Grubbs at α = 0.05.
- Without it M = 10.1, SD = 0.172 (with it M = 11.9, SD = 5.29).
- The value stays in the table until you exclude it.
Kaplan–Meier survival analysis (survival)
How to draw Kaplan–Meier survival curves and compare them in Dioptra: enter times as X and events (1) or censoring (0) per group in a survival table, press Analyze (Ctrl-Shift-A) and choose Survival curves (Kaplan–Meier).
- When: time to an event with censoring, one or more groups.
- Tables: survival (X = time; each data set 1 = event, 0 = censored).
- Options:
compare"log_rank"(also"gehan_breslow_wilcoxon", which leads the sentence; both are always shown),ci_level0.95. - Checks: values used, event codes (the bad cell is named), number of events, proportional hazards (stated, not tested), uninformative censoring, independence.
- Outputs: a Kaplan–Meier table per group (time, at risk, events, censored, percent
survival, SE, CI), Median survival with the Brookmeyer–Crowley CI, Curve comparison
(log-rank, Gehan–Breslow–Wilcoxon, trend for three or more groups), Observed and expected
events, Hazard ratio both ways (Mantel–Haenszel and log-rank methods). Graph:
kaplan_meier(step curves, censor ticks, CI bands, numbers at risk). - Method: survival.
- Example:
- Data: Gehan’s leukaemia trial, 6-MP (21 patients, 9 relapses) and placebo (21, 21).
- Median survival 23 weeks with 6-MP (95% CI 13 to undefined), 8 with placebo (95% CI 4 to 11).
- Log-rank χ²(1) = 16.79, p < .001.
- Hazard ratio (6-MP / placebo, Mantel–Haenszel) 0.19 (95% CI 0.09 to 0.43).
ROC curve (roc)
How to draw a ROC curve in Dioptra: enter controls and patients as two data sets of a column table, press Analyze (Ctrl-Shift-A) and choose ROC curve.
- When: how well a measurement separates patients from controls; choose a cut-off.
- Tables: column (controls first, then patients), variables (a score variable and a
category variable with
positive_label). - Options:
columns(first two),positive_labelnull,ci_level0.95. - Checks: direction (AUC < 0.5 means the columns are the wrong way round), sample size, representative samples stated, independence.
- Outputs: Area under the curve (DeLong and Hanley–McNeil SEs and CIs), test of AUC =
0.5, Cut-offs (sensitivity and specificity with Wilson CIs, LR+, LR−, Youden J), Best
cut-offs (Youden, closest to (0, 1)). Graph:
roc. - Method: roc.
- Example:
- Data: 10 controls (1.2 … 2.0) and 10 patients (3.4 … 5.2).
- AUC = .90 (95% CI .77 to 1.00, DeLong), p < .001 against chance.
- Youden cut-off ≥ 3.3: sensitivity 80.0%, specificity 90.0%.
- Direction check passes.
Bland–Altman analysis (bland_altman)
How to run a Bland–Altman analysis in Dioptra: enter the two methods as two data sets of a column table, one subject per row, press Analyze (Ctrl-Shift-A) and choose Bland–Altman.
- When: agreement between two methods measured on the same subjects.
- Tables: column, variables (two data sets paired by row).
- Options:
columns(first two),method"difference"(also"ratio","percent_difference"),ci_level0.95. - Checks: values used (unpaired rows dropped), sample size (30 pairs), normality of the differences, proportional bias, acceptable agreement stated, independence.
- Outputs: Bias and limits of agreement (bias, SD, lower and upper limits, each with
CI), Proportional bias (slope of the difference on the mean), Normality. Graph:
bland_altman(differences),scatterwith bias and limit lines (ratio, percent). - Method: agreement
bland_altman. - Example:
- Data: the ROC example’s two columns read as methods A and B.
- Bias (A − B) -1.6 (95% CI -2.3 to -0.9).
- Limits of agreement -3.4 (95% CI -4.6 to -2.2) to 0.20 (95% CI -0.96 to 1.36).
- No proportional bias (p = .458).
Principal component analysis (PCA) (pca)
How to run a principal component analysis in Dioptra: enter the variables in a Multiple variables table, press Analyze (Ctrl-Shift-A) and choose Principal component analysis.
- When: summarise several correlated numeric variables by a few components.
- Tables: variables.
- Options:
columns[] (all numeric),standardizetrue (correlation matrix),componentsnull (chosen by Horn’s parallel analysis). - Checks: values used (incomplete rows dropped), rows per variable (5 or more), comparable scales when not standardized, linear relationships stated, independence.
- Outputs: Eigenvalues (% of variance, cumulative %, parallel-analysis 95th percentile,
retained), Loadings, Eigenvectors, Scores. Graph:
scatter(scores on PC1 and PC2);columngives the scree plot. - Method: multivariate.
- Example:
- Data: three variables on 10 cases, two of them strongly correlated with each other and negatively with the third.
- The first component explains 97.1% of the variance.
- Parallel analysis and Kaiser’s rule both retain one component.
Statistical methods and references
One line per public function of dioptra-stats: what it computes, the reference it
implements, and the conventions that differ between Prism and SciPy. Conventions that hold
everywhere:
NaNis a missing value. Descriptive statistics and tests skip it; paired tests drop incomplete pairs; outlier indices keep the caller’s positions. Infinities are rejected withStatsError::InvalidArgument.- Two-sample differences are
x - y. Prism shows “B - A” (later column minus earlier); the presentation layer flips the sign where it mimics Prism. - Every test returns the two-sided p and both one-sided p-values (
p_less,p_greater) pluspfor the requested alternative. Prism’s “one-tailed” p isp_lessorp_greaterfor the predicted direction. - Upper tails are computed directly, never as
1 - cdf; quantiles invert whichever tail is smaller. - The low-level
specialkernels returnNaNfor invalid arguments; distribution constructors and tests returnStatsError.
special (special functions)
| Function | Reference | Notes |
|---|---|---|
ln_gamma(x) | fdlibm lgamma_r (via libm) | ln abs Gamma(x), about 1 ulp. |
gamma(x) | musl tgamma (via libm) | |
digamma(x) | Abramowitz & Stegun 6.3.5, 6.3.7, 6.3.18 | Recurrence to x >= 10, asymptotic series; reflection for x < 0. |
ln_beta(a, b) | R lbeta (Ihaka) | Stirling-correction form when max(a,b) >= 8, no cancellation at a = b = 5000. |
beta(a, b) | exp(ln_beta) | |
ln_choose(n, k) | via ln_beta | Real n, k. |
erf(x), erfc(x) | fdlibm (via libm) | erfc keeps relative accuracy in the tail. |
log1p(x), expm1(x) | IEEE (std) | |
log1pmx(x) | R log1pmx (Welinder) | ln(1+x) - x without cancellation. |
stirling_error(z) | Loader (2000), R stirlerr; A&S 6.1.40 | ln Gamma(z) minus Stirling’s formula. |
binomial_deviance(x, np) | Loader (2000), R bd0 | |
binomial_pmf_raw(x, n, p, q) | Loader (2000), R dbinom_raw | Saddle-point binomial pmf, q supplied separately. |
ln_gamma_prefactor(a, x) | Temme (1987); DiDonato & Morris (1986) rcomp | ln(x^a e^-x / Gamma(a)), Stirling form for a >= 8. |
gammainc_pair(a, x) | Numerical Recipes 3rd ed. §6.2; DiDonato & Morris (1986), TOMS 654 | (P, Q): series for x < a+1, Lentz continued fraction for Q otherwise. |
gammainc_p, gammainc_q | as gammainc_pair | Regularised lower / upper incomplete gamma. |
gammainc_inv_pq(a, p, q), gammainc_inv, gammaincc_inv | DiDonato & Morris (1986) §5; Wilson & Hilferty (1931) start | Bracketed Halley iteration on the smaller tail. |
ln_beta_prefactor(a, b, x, y) | DiDonato & Morris (1992), TOMS 708 brcomp; Loader (2000) | ln(x^a y^b / B(a,b)), expanded around the mean when a, b >= 8. |
betainc_pair_xy(a, b, x, y) | Numerical Recipes 3rd ed. §6.4 (modified Lentz); TOMS 708 ideas (symmetry, bpser, brcomp) | (I, 1 - I) with y = 1 - x supplied; switch at x = (a+1)/(a+b+2). |
betainc_pair, betainc, betaincc | as above | Regularised incomplete beta and its complement. |
betainc_inv_pq(a, b, p, q), betainc_inv, betaincc_inv | Numerical Recipes 3rd ed. §6.14 start (A&S 26.5.22); bracketed Halley | Returns (x, 1 - x); solves for the smaller of x and 1 - x. |
norm_quantile(p), norm_quantile_upper(q), norm_quantile_pq | Wichura (1988), AS 241 PPND16 | About 1e-16; R’s qnorm algorithm. Upper tail by symmetry, so tiny q is exact. |
norm_quantile_ppnd7(p) | Wichura (1988), AS 241 PPND7 | Single-precision variant, kept for reference. |
erfc_inv(y), erf_inv(x) | via AS 241 | Central range uses the PPND16 rational directly (no 1 + x rounding). |
dist (distributions)
Each continuous distribution implements Continuous (pdf, cdf, sf, ppf, isf) and
has a cdf_sf or quantile_pq method taking both tails.
| Function | Reference | Notes |
|---|---|---|
std_normal_pdf, std_normal_ln_pdf | R dnorm argument splitting | Full relative accuracy for large z. |
std_normal_cdf, std_normal_sf | erfc (fdlibm) with argument-rounding correction; Laplace continued fraction for Mills’ ratio beyond 37 (A&S 26.2.14) | Relative accuracy to the underflow limit. |
std_normal_ln_cdf, std_normal_ln_sf | ln phi + ln Mills’ ratio for z < -5 | |
Normal | as above; AS 241 quantile | |
StudentT (cdf_sf, two_sided_p, quantile_pq) | A&S 26.7.1 via the incomplete beta | Non-integer df; df = inf is the normal. |
ChiSquared, Gamma | regularised incomplete gamma | |
FisherF | A&S 26.6.2 via the incomplete beta | Beats SciPy’s f.isf in the far upper tail. |
Beta | regularised incomplete beta | |
Binomial | Loader (2000) pmf; A&S 26.5.24 cdf/sf | |
Hypergeometric | R dhyper (Loader kernel), R pdhyper summation | SciPy’s hypergeom(M, n, N) parameter order: population, successes, draws. |
NoncentralT | Lenth (1989), AS 243; Guenther-style reflection for t < 0 | Tails below 1e-6 (and nc > 37 or df > 4e5) by adaptive Gauss–Kronrod quadrature of integral Phi(tu - delta) f(u) du, which keeps relative accuracy where AS 243 and SciPy’s nct fail (x = -3, df = 5, nc = 8: true 1.90e-20, SciPy 8.0e-20). |
StudentizedRange | Copenhaver & Holland (1988); Lund & Lund (1983), AS 190 | Inner composite Gauss–Legendre, outer adaptive Gauss–Kronrod over the chi scale; upper tail integrated directly. About 1.2-1.5 ms per cdf. |
Dunnett (new, balanced, from_group_sizes, cdf_sf, p_value, quantile) | Dunnett (1955); product-correlation reduction rho_ij = lambda_i lambda_j | Deterministic double integral (not SciPy’s randomized QMC, which is reproducible only to about 1e-3). Two-sided and one-sided; balanced and unbalanced. |
p_two_sided_t, p_less_t, p_greater_t | Student t tails | |
p_two_sided_z, p_less_z, p_greater_z | normal tails | |
p_upper_chi2, p_upper_f | upper tails, computed directly | |
t_critical_two_sided(alpha, df) | t quantile |
descriptive
| Function | Reference | Notes |
|---|---|---|
compensated_sum | Neumaier (1974) | |
clean, count | Drop NaN; reject infinities. | |
sum, mean | Higham (2002) two-pass corrected mean | |
variance, sd, sem, sd_population, cv | Chan, Golub & LeVeque (1983) corrected two-pass | n - 1 denominators (Prism, SciPy ddof=1); NumPy’s default is n. CV is a fraction (Prism shows %). |
min, max, range, median | ||
quantile, quantile_sorted, percentile, quartiles | Hyndman & Fan (1996) | QuantileMethod::Prism (type 6, (n+1)p, the default) vs Linear (type 7, SciPy/NumPy/Excel PERCENTILE.INC) vs TukeyHinges (R fivenum, for box plots). |
skewness_g1, kurtosis_g2 | moment estimators (SciPy bias=True) | |
skewness, kurtosis | Joanes & Gill (1998) G1, G2 | Bias-corrected, what Prism and Excel report; excess kurtosis. |
geometric_mean, geometric_sd_factor, ci_geometric_mean | Kirkwood (1979) | All values > 0. |
harmonic_mean | All values > 0. | |
ci_mean | t interval | Any level. |
lag1_autocorrelation | NIST StRD definition | Corrected for the rounding of the mean. |
Summary::compute | all of the above | Prism’s column statistics in one pass. |
ttest
| Function | Reference | Notes |
|---|---|---|
one_sample | Student (1908) | Estimate and CI are of mean - mu (SciPy’s CI is of the mean). |
unpaired | pooled-variance t | |
welch | Welch (1947); Satterthwaite (1946) df | Fractional df. Dioptra’s default unpaired test (DESIGN §2); Prism’s default is the pooled test. |
paired | one-sample t on x - y | Cohen’s d is d_z. |
ratio_paired | GraphPad “ratio paired t test” | Paired t on log10(x) - log10(y); geometric mean ratio and back-transformed CI. |
hedges_j(df) | Hedges & Olkin (1985) | Exact J = Gamma(df/2)/(sqrt(df/2) Gamma((df-1)/2)); hedges_g_approx uses 1 - 3/(4df - 1). Two-sample effect sizes use the pooled SD and n1 + n2 - 2 df, also for Welch. R^2 = t^2/(t^2 + df). |
rank, mannwhitney, wilcoxon
| Function | Reference | Notes |
|---|---|---|
rank_with_ties, tie_term | midranks (SciPy rankdata('average')) | Tie sizes of groups >= 2; sum(t^3 - t). |
u_distribution(m, n) | Mann & Whitney (1947) recurrence (Löffler 1983) | Exact null distribution of U, additions only. |
mann_whitney | Mann & Whitney (1947); Hodges & Lehmann (1963); Moses CI (Hollander & Wolfe 1999 §4.3) | U1 is SciPy’s statistic, u = min(U1, U2) is Prism’s. Exact when there are no ties and n1 n2 <= 10,000; tie-corrected normal approximation with optional 0.5 continuity correction otherwise. Prism computes an exact p even with ties (up to 100 per group); we switch to the asymptotic method and say so in method. |
signed_rank_distribution(n) | generating function prod(1 + q^i) | |
signed_rank, paired, one_sample (wilcoxon) | Wilcoxon (1945); Pratt (1959) zeros; Cureton (1967) variance | ZeroMethod::Wilcoxon (drop zeros, Prism’s default) or Pratt. Exact for n <= 50 without ties or zeros; otherwise normal approximation with tie (and Pratt-zero) adjustment; continuity correction off by default (SciPy’s default). Hodges–Lehmann pseudo-median of the Walsh averages (zeros kept, and a zeros-dropped variant) with Tukey’s CI. Prism computes exact p with ties below 200 pairs; we use the asymptotic method there and say so. |
normality
| Function | Reference | Notes |
|---|---|---|
shapiro_wilk | Royston (1992); Royston (1995), AS R94 | 3 <= n <= 5000. Normal scores from AS 241 PPND16 (as R’s swilk.c). |
dagostino_pearson | D’Agostino (1970); Anscombe & Glynn (1983); D’Agostino, Belanger & D’Agostino (1990) | Exactly SciPy’s normaltest; n >= 8. |
anderson_darling | Anderson & Darling (1954); Stephens (1974); D’Agostino & Stephens (1986) Table 4.9 | A*^2 = A^2 (1 + 0.75/n + 2.25/n^2); p from Table 4.9 (R nortest, Prism). Also SciPy’s critical values. SciPy >= 1.17 method='interpolated' p is a different method. |
ks_one_sample, kolmogorov_smirnov_normal | Kolmogorov (1933); exact p by kolmogn_sf | Parameters specified a priori. |
kolmogn_sf(n, d) | Simard & L’Ecuyer (2011) method selection; Durbin (1968) matrix as Marsaglia, Tsang & Wang (2003); Pelz & Good (1976) | SciPy kstwo uses Pomeranz (1974) for 0.754693 < n d^2 <= 4; Durbin’s matrix is exact there too. |
smirnov_sf(n, d) | Birnbaum & Tingey (1951) | One-sided exact tail. |
kolmogorov_sf(z) | Kolmogorov (1933) limiting series | SciPy kstwobign. |
lilliefors | Lilliefors (1967) | Estimated mean and SD (n - 1). Two p-values: p_table (statsmodels simulation table, 10^7 replications, reproduced in lilliefors_table.rs, BSD-3) and p_dallal_wilkinson (Dallal & Wilkinson 1986 when <= 0.1, else the table, as statsmodels pvalmethod='approx'). Prism uses Dallal–Wilkinson and reports “P > 0.10” above. |
lilliefors_p_table, lilliefors_p_dallal_wilkinson | as above |
outliers
| Function | Reference | Notes |
|---|---|---|
grubbs(data, alpha) | Grubbs (1969); NIST/SEMATECH e-Handbook §1.3.5.17 | Two-sided, iterated while significant; G uses the SD of all remaining values; p is the Bonferroni bound 2n P(T > t_obs). |
rout_column(data, q) | Motulsky & Brown (2006), BMC Bioinformatics 7:123; GraphPad guide | Lorentzian fit of a constant with adaptive RSDR = P68 N/(N-K) (Prism’s guide; some renderings of the paper show sqrt(N/(N-K))); FDR flagging of the farthest 30% at Q on N-K df. rsdr_alt_mad (1.4826 MAD) is reported for comparison only. |
rout_fdr_flags(resid, rsdr, k, q) | Motulsky & Brown (2006) eqs. 17-18 | The FDR step shared by the column and curve versions: ranks abs(r), tests the farthest 30% on N-K df, flags the first significant point and every point farther out. |
rsdr(resid, k) | GraphPad Curve Fitting Guide | P68 of abs(residuals) (Hyndman-Fan type 7) times N/(N-K). |
rout_curve(eq, data, q, opts) | Motulsky & Brown (2006) | ROUT for a fitted curve; delegates to nonlinear::rout_fit. |
quad, roots (numerical infrastructure)
| Function | Reference | Notes |
|---|---|---|
gauss_legendre, gauss_legendre_cached, composite_nodes | Numerical Recipes 3rd ed. §4.6 gauleg | |
integrate | QUADPACK QAG with QK15 (Piessens et al. 1983) | Raw |
integrate_pair | QUADPACK QAG/QK15 with QUADPACK’s error scaling | Two integrands sharing nodes (a cdf and its complement). |
halley_bracketed | Numerical Recipes 3rd ed. §9.4 rtsafe with Halley correction | Bisection (geometric across decades) when a step leaves the bracket. |
brent, expand_bracket | Brent (1973), zeroin |
anova
Sums of squares are computed on data shifted by the first observation (exact by Sterbenz’s lemma when values share leading digits) with two-pass means and compensated sums.
| Function | Reference | Notes |
|---|---|---|
one_way_anova(groups) | Fisher (1925); omega^2 Hays (1963) | SS/df/MS/F/p, R^2 = eta^2, omega^2, residual SD, group n/mean/SD/SEM, plus the five tests below as Options (None when a group has n < 2 or zero variance). |
welch_anova(groups) | Welch (1951), doi:10.2307/2332579 | w_i = n_i/s_i^2; df2 = (k^2-1)/(3 Lambda). statsmodels anova_oneway(use_var='unequal'), R oneway.test. |
brown_forsythe_anova(groups) | Brown & Forsythe (1974), doi:10.2307/1267501; Mehrotra (1997), doi:10.1080/03610919708813431 | Test of means: F* = SS_b / sum (1 - n_i/N) s_i^2, Satterthwaite df2. p uses k - 1 numerator df (the 1974 test, Prism’s “Brown-Forsythe ANOVA”); mehrotra_p uses Mehrotra’s df (statsmodels default). |
bartlett(groups) | Bartlett (1937); Snedecor & Cochran (1989) §13.9 | SciPy bartlett. |
levene(groups, center) | Levene (1960); Brown & Forsythe (1974), doi:10.1080/01621459.1974.10482955 | Median = Brown–Forsythe test of variances (Prism’s variance check), Mean = Levene. SciPy levene. |
cells_from_long(a, b, y) | Long format (level indices) to the [row][col] -> values cell array. | |
two_way_anova(cells, opts) | Searle (1987); Langsrud (2003), doi:10.1023/A:1023260610025 | Sum-to-zero coded GLM, refined Householder QR; type I/II/III SS as nested-model RSS differences = statsmodels anova_lm(typ=1/2/3) with C(., Sum). Type III (default) is Prism’s; treatment-coded type III is wrong and not offered. Empty cells allowed where estimable (df = rank added). No replicates: interaction dropped (additive model). Partial eta^2 per term. |
rm_anova(m) | statsmodels AnovaRM; Box (1954); Greenhouse & Geisser (1959), doi:10.1007/BF02289823; Huynh & Feldt (1976) with Lecoutre (1991), doi:10.3102/10769986016004371; Mauchly (1940), doi:10.1214/aoms/1177731915 | Subjects x conditions; listwise deletion (count reported). GG epsilon from orthonormal Helmert contrasts (and Box’s formula as a cross-check), HF capped at 1 (uncapped also reported), lower bound; corrected df and p; Mauchly W, chi-square, df, p; F for subjects. Prism reports GG only. |
kruskal_wallis(groups) | Kruskal & Wallis (1952), doi:10.1080/01621459.1952.10483441 | Tie-corrected H, chi-square p (SciPy kruskal); mean ranks; epsilon^2. Prism’s small-sample exact p is not implemented. |
friedman(m) | Friedman (1937), doi:10.1080/01621459.1937.10503522; Kendall & Babington Smith (1939) | Within-subject midranks, tie-corrected statistic (SciPy friedmanchisquare), Kendall’s W = chi2/(n(k-1)); listwise deletion. |
posthoc
Comparison (i, j) estimates value_i - value_j; all-pairs lists i < j; many-to-one lists (treatment, control). statsmodels’ Tukey reports mean_j - mean_i (opposite sign).
| Function | Reference | Notes |
|---|---|---|
p_adjust(p, method) | Bonferroni; Šidák (1967); Holm (1979); Hochberg (1988); Hommel (1988); Benjamini & Hochberg (1995); Benjamini & Yekutieli (2001); Benjamini, Krieger & Yekutieli (2006) | statsmodels multipletests (holm-sidak is step-down Šidák; TwoStageBky{alpha} = fdr_tsbky, one iteration, depends on alpha). Šidák via expm1/log1p (exact for tiny p). NaNs pass through and do not count. |
Adjust::name() | Display name. | |
tukey_kramer(groups, alpha) | Tukey (1953); Kramer (1956), doi:10.2307/3001469 | q = |
dunnett(groups, control, alternative, alpha) | Dunnett (1955), doi:10.1080/01621459.1955.10501294 | Exact multivariate-t p and critical value (dist::Dunnett, deterministic double integral), two- or one-sided, simultaneous CIs. |
pairwise_t(groups, pairs, adjust, alpha) | Fisher (1935) LSD; Šidák (1967); Holm (1979) | Pooled-variance t tests on chosen pairs (all if None); CIs for none/Bonferroni/Šidák only (as Prism). |
dunn_kruskal(groups, control, adjust, alpha) | Dunn (1964), doi:10.1080/00401706.1964.10490181; Daniel, Applied Nonparametric Statistics | Pooled midranks, tie-corrected SE; all pairs or vs control. |
dunn_friedman(m, control, adjust, alpha) | Prism’s documented post test after Friedman | z = (Rbar_i - Rbar_j)/sqrt(k(k+1)/(6n)), no tie correction. |
compact_letters(k, comparisons) | Piepho (2004), J. Comput. Graph. Stat. 13:456 | Insert-absorb letter display from the significant flags. |
contingency
2 x 2 layout [[a, b], [c, d]]: rows = groups, columns = outcome (event, no event).
| Function | Reference | Notes |
|---|---|---|
chi_square_independence(table) | Pearson (1900); Yates (1934); Wilks (1938) G; Haberman (1973) residuals; Cochran (1954) rule | Pearson, Yates (2 x 2 only, |
chi_square_gof(observed, proportions, ddof) | Pearson (1900) | SciPy chisquare; proportions normalised; df = k - 1 - ddof. |
fisher_exact_2x2(table, ci_level) | Fisher (1935); Cornfield (1956) | Two-sided = sum of P <= P_obs (1 + 1e-7) (R, SciPy, Prism); one-sided tails; conditional MLE odds ratio and exact CI by Brent on ln(OR) (R fisher.test, SciPy odds_ratio(kind='conditional')). |
risk_measures(table, ci_level) | Woolf (1955), doi:10.1111/j.1469-1809.1955.tb01285.x; Katz et al. (1978), doi:10.2307/2530610 | OR with Woolf CI, RR with Katz log CI (None with zero cells), risk difference with Wald CI. Prism’s Koopman (RR) and Baptista–Pike (OR) intervals are not implemented. |
fisher_exact_rxc(table) | Freeman & Halton (1951); Mehta & Patel (1983), doi:10.1080/01621459.1983.10477989 | Network algorithm: column stages, nodes = multisets of remaining row totals, merged path values, closed-form node mass, relaxation bounds; exact p, observed-table P, table count. Limits FISHER_RXC_WORK_LIMIT, FISHER_RXC_PATH_LIMIT (NoConvergence beyond; see CORRECTNESS.md). |
mcnemar(table) | McNemar (1947), doi:10.1007/BF02295996; Edwards (1948) | Chi-square with and without continuity correction (Prism uses the corrected one), exact binomial p (statsmodels). |
cochran_armitage(successes, totals, scores) | Cochran (1954), doi:10.2307/3001616; Armitage (1955), doi:10.2307/3001775 | z without the N/(N-1) factor (chi2 = z^2 = R prop.trend.test, Prism’s “chi-square for trend”); the N/(N-1) variant reported too. |
wilson_ci(successes, n, level) | Wilson (1927) | Score interval, no continuity correction. |
diagnostic(table, ci_level) | Wilson (1927); Simel, Samsa & Matchar (1991) | [[TP, FP], [FN, TN]]: sensitivity, specificity, PPV, NPV with Wilson CIs; LR+/LR- with log CIs. |
correlation
| Function | Reference | Notes |
|---|---|---|
pearson(x, y, ci_level) | Pearson (1896); Fisher (1921) z | Two-pass centred sums; t test on n - 2 df; Fisher-z CI (Prism). |
spearman(x, y, ci_level) | Spearman (1904); Bonett & Wright (2000) CI | Pearson on midranks (correct with ties); t-approximation p (SciPy); exact permutation p for n <= 9 (SPEARMAN_EXACT_MAX_N), ties kept. R uses AS 89 for larger n without ties: not implemented. |
kendall(x, y) | Kendall (1945); Knight (1966), doi:10.1080/01621459.1966.10480879 | tau-b and tau-c in O(n log n); tie-corrected asymptotic p (SciPy); exact p without ties for n <= 170 from the inversion distribution; p_auto = SciPy method='auto'. |
kendall_exact_two_sided(n, d) | Kendall (1970) ch. 4 | 2 P(D <= min(d, n0 - d)), Mahonian recurrence in probabilities. |
correlation_matrix(columns, method, adjust) | Pairwise deletion; p adjusted over the k(k-1)/2 pairs. | |
partial_correlation(x, y, z, ci_level) | Fisher (1924) | Correlation of residuals on z (refined QR), t on n - 2 - q df. |
regression
| Function | Reference | Notes |
|---|---|---|
simple_linear_regression(x, y, opts) | least squares; Cook (1977); Durbin & Watson (1950); Swed & Eisenhart (1943), doi:10.1214/aoms/1177731178; Draper & Smith (1998) §2.1 | Coefficients and SEs from the refined QR; R^2 (uncentred through the origin, as R/statsmodels/NIST; r_squared_centered is Prism’s), F, Sy.x, x-intercept, slope vs hypothesised value, leverage, studentised residuals, Cook’s D, Durbin–Watson (input order), runs test (x order, exact zeros dropped), replicates (lack-of-fit) F. |
SimpleRegression::bands(x0) | Confidence and prediction bands at x0. | |
runs_test_by_x(x, resid), runs_test(signs) | Wald & Wolfowitz (1940); Swed & Eisenhart (1943) | Runs, E[R], z (no continuity), exact P(R <= runs) (Prism’s one-sided p). |
runs_cdf(r, n1, n2) | Swed & Eisenhart (1943) | Exact cdf of the number of runs. |
multiple_regression(x_rows, y, opts) | Golub (1965); Björck (1967), BIT 7:257; Marquardt (1970) VIF | Equilibrated pivoted Householder QR with double-double augmented-system refinement; aliased columns reported (R lm convention, in column order); SEs from refined (X’X)^{-1}; ANOVA table; adj R^2; VIFs; covariance. statsmodels OLS. |
polynomial_regression(x, y, degree, opts) | as above | Powers x^k formed in double-double so the refinement sees the exact design of the double inputs (NIST Filip LRE 14.0). |
deming_regression(x, y, lambda, ci_level) | Deming (1943); Linnet (1993), doi:10.1093/clinchem/39.3.424 | lambda = sigma_y^2/sigma_x^2 (Prism input: (SD_y/SD_x)^2); jackknife SEs (NCSS, R mcr); t(n-2) CIs. Prism’s SE method is undocumented. |
equations (curve-fit equation library)
Prism’s equation and parameter names (GraphPad Curve Fitting Guide, “Models built in to
Prism”). Each Equation holds ParamSpecs (name, InitialRule, default bounds, data-set
constant flag), DerivedSpecs and its XTransform. Gradients are hand-written for the
simple models and exact forward-mode automatic differentiation for the rest (5PL,
bell-shaped, two-site competition, fit Ki, [Agonist] variable slope, log-logistic,
growth, power series); equations::tests::gradients_match_finite_differences checks every
one against Richardson differences.
| Function | Reference | Notes |
|---|---|---|
Equation::custom(expr), Equation::custom_with(expr, vars, params) | Pratt (1973) top-down operator precedence; Griewank & Walther (2008) forward AD | Parses + - * / ^ (**), unary minus below ^, exp ln log log10 log2 sqrt abs sin cos tan asin acos atan sinh cosh tanh pow, pi, e; log is base 10 (Prism); identifiers case-insensitive; parameters are the free identifiers in order of appearance (or the given order). Exact derivatives. |
Equation::from_closure(name, params, f) | Rust closure; central-difference gradient. | |
Equation::eval, eval_grad, eval_grad_fd(.., richardson) | Central differences with h = eps^(1/3) max(abs(p), 1), or Richardson from h and h/2 at h = eps^(1/5) max(abs(p), 1). | |
Equation::initial_values(x, y), InitialRule::eval, DataSummary::new | GraphPad “Rules for initial values” | YMIN, YMAX, YMID, XMIN, XMAX, XMID, X at YMID (linear interpolation of the replicate means), Y at XMIN/XMAX, SIGN(trend), rate ln 2/half-time, LS slope/intercept, ranges, or a function. |
Equation::derived_values(p) | EC50 = 10^LogEC50, IC50, Ki, half-life ln 2/K, tau 1/K, doubling time, spans, Intercept2, Gaussian area. | |
straight_line, line_through_origin, polynomial(1..=6), segmental_linear | Lines and polynomials are linear: fitted from the exact linear least-squares solution. | |
one_phase_decay, two_phase_decay, plateau_then_one_phase_decay, one_phase_association, two_phase_association, plateau_then_one_phase_association, exponential_growth | Prism exponential family | Y0, Plateau, K (PercentFast, KFast, KSlow; X0). |
log_agonist_vs_response, log_agonist_variable_slope (4PL), log_agonist_normalized_variable_slope, log_inhibitor_vs_response, log_inhibitor_variable_slope, asymmetric_5pl, agonist_vs_response, agonist_variable_slope, bell_shaped, log_logistic | Prism dose-response; Gottschalk & Dunn (2005) 5PL | X = log10(concentration) where x_transform is Log10; logistic pieces evaluated without overflow. |
michaelis_menten, substrate_inhibition, allosteric_sigmoidal, competitive_inhibition | Prism enzyme kinetics | I (inhibitor) is a data-set constant. |
one_site_specific, specific_binding_hill, one_site_total, one_site_fit_logic50, one_site_fit_ki, two_sites_fit_logic50 | Prism binding; Cheng & Prusoff (1973) | HotNM, HotKdNM are data-set constants in Fit Ki. |
gaussian, lorentzian, sine_wave, sine_wave_baseline, power, power_series, boltzmann_sigmoid, logistic_growth, gompertz_growth | Prism Gaussian, sine, classic and growth equations | |
library(), by_name(name) | Every built-in equation; lookup by Prism name or alias (4PL, 3PL, 5PL, line, MM). | |
cheng_prusoff_ki(ic50, ligand, kd) | Cheng & Prusoff (1973), Biochem Pharmacol 22:3099 | IC50/(1 + [L]/Kd). |
nonlinear (curve fitting)
| Function | Reference | Notes |
|---|---|---|
fit(eq, data, opts), fit_global(eq, datasets, opts) | Moré (1978), doi:10.1007/BFb0067700; MINPACK lmder (Moré, Garbow & Hillstrom 1980) | Levenberg–Marquardt with scaled trust region (diag from Jacobian column norms, non-decreasing), pivoted QR, Moré lambda update; ftol = xtol = 1e-12, gtol 1e-15, 1000 iterations. Then Newton polishing (J’r compensated, Hessian J’J + sum r_i H_i with H_i from central differences of the exact Jacobian) accepted while the gradient norm falls. Box constraints: active-set projection (a parameter on a bound whose descent direction leaves the box is held for the iteration; the step is projected and its predicted reduction recomputed); at-bound parameters get no SE and flag “hit constraint”. Fixed, per-data-set constant and shared parameters (global fit); per-data-set local parameters. SEs sqrt(s^2 (J’J)^{-1}) from the refined QR; t CIs; covariance and correlation; dependency 1 - 1/(C_jj (J’J)_jj); “ambiguous” when a column is aliased or a dependency > 0.9999. SS, df, Sy.x, R^2 = 1 - SS/SS_tot (weighted with the final weights), adjusted R^2 = 1 - (1-R^2)(N-1)/(N-k), AICc with K = k + 1 (Prism). Runs test (x order) and replicates (lack-of-fit) test per data set. Residuals Y - f. |
FitOptions::weighting | Prism Method tab | None, 1/Y, 1/Y^2, 1/Y^K, 1/X, 1/X^2, 1/SD^2, Poisson (deviance residuals, unscaled covariance). WeightBase::Curve (Prism) weights by the fitted Y with the full derivative of (y - f)/abs(f)^{K/2}; Data uses fixed 1/abs(y)^K (SciPy sigma). |
FitMethod::Robust | Motulsky & Brown (2006) | Adaptive Lorentzian: start from the LS fit; alternate RSDR = P68 N/(N-K) of the current residuals with an LM minimisation of sum ln(1 + (r/RSDR)^2) (residuals transformed to RSDR sign(r) sqrt(ln(1 + (r/RSDR)^2)) so the LM objective is the merit) until parameters and RSDR change < 1e-10 (max 200 rounds). No SEs (as Prism). |
FitMethod::RobustFixedScale { loss, scale } | SciPy least_squares | Cauchy c^2 ln(1 + (r/c)^2) or soft-L1 2c^2 (sqrt(1 + (r/c)^2) - 1); robust_objective = sum. |
FitOptions::profile_ci | Venzon & Moolgavkar (1988), doi:10.2307/2347496; Bates & Watts (1988) ch. 6 | Prism’s “asymmetrical” CI: the two values of each parameter where the re-minimised SS = SS_min (1 + F(level; 1, df)/df); bracketed by steps of 1.6^k SE (at most 40 or the bound), then Brent. +/-inf if never reached; the bound if a constraint is reached first. Equals the t CI for linear models. |
FitOptions::jacobian | Exact (analytic/AD; closures: central differences) or FiniteDifference { richardson } for any equation. | |
FitOptions::outlier_q, rout_fit(eq, data, q, opts), rout_fit_global | Motulsky & Brown (2006) | Robust fit, RSDR, FDR test on the weighted residuals pooled over data sets with N - K df, least-squares refit without the outliers (their residuals still reported). |
bands(eq, fit, d, xs, opts), FitOptions::curve_points | Delta method; Seber & Wild (1989) §5.1 | Confidence band f +/- t sqrt(g’ Cov g), prediction band f +/- t sqrt(g’ Cov g + s^2/w(x)) (w from the curve for 1/Y^K; 1 for 1/SD^2). |
compare_summaries(simple, complex, alpha), compare_models(simple, complex), FitSummary::aicc | Motulsky & Christopoulos (2004) ch. 21-22; Burnham & Anderson (2002) | Extra-sum-of-squares F = ((SS1-SS2)/(df1-df2))/(SS2/df2); AICc with Prism’s K; delta = AICc(simple) - AICc(complex); Akaike weights; preferred model by each. |
fit_family(eq, datasets, opts) | Prism “one curve for all data sets?” | Separate fits (sharing removed) versus the global fit; F test and AICc with summed SS, df and parameters. |
CurveFit::values, params, se, summary | Accessors. |
survival
| Function | Reference | Notes |
|---|---|---|
kaplan_meier(time, event, opts) | Kaplan & Meier (1958); Greenwood (1926); Kalbfleisch & Prentice (1980) | Events before censorings at tied times; rows at distinct event times; Greenwood variance; CIs log-log (Prism “asymmetrical”, lifelines; default), linear clipped (Prism “symmetrical”), log (R default); undefined once S = 0. Median = first t with S <= 0.5 (+inf if never); Brookmeyer & Crowley (1982) CI by inverting the chosen band. Step curve and censor marks. |
KaplanMeier::survival_at, ci_at, at_risk_at | Right-continuous lookups; at risk = subjects with time >= t. | |
logrank(groups, weight, scores) | Mantel (1966); Gehan (1965)/Breslow (1970); Tarone & Ware (1977); Fleming & Harrington (1991) ch. 7 | k groups, hypergeometric variance, chi2 = U’V^{-1}U on k-1 df; weights 1, n, sqrt(n); O/E per group; test for trend with scores (default 1..k). lifelines logrank_test. |
hazard_ratio(g1, g2, ci_level) | GraphPad “The hazard ratio”; Berry, Kitchin & Mock (1991) | Log-rank method (O1/E1)/(O2/E2) with SE sqrt(1/E1 + 1/E2); Mantel–Haenszel exp((O1-E1)/V) with SE 1/sqrt(V). Group 1 relative to group 2. |
cox_ph(time, event, covariates, ties, ci_level) | Cox (1972); Efron (1977); Breslow (1974) | Newton–Raphson with step halving on the partial likelihood (covariates centred); SEs from the inverse information; Wald z, p, HR CI; likelihood-ratio test. Matches the 50-digit maximum where lifelines stops early. |
cox_log_likelihood(time, event, covariates, beta, ties) | Partial log-likelihood at a given beta. |
roc
| Function | Reference | Notes |
|---|---|---|
roc(positives, negatives, opts), roc_from_labels(scores, labels, opts) | DeLong, DeLong & Clarke-Pearson (1988), doi:10.2307/2531595; Sun & Xu (2014), doi:10.1109/LSP.2014.2337313; Hanley & McNeil (1982); Youden (1950) | Positive if score >= threshold; thresholds +inf then every distinct score descending (sklearn drop_intermediate=False). Sensitivity, specificity with Wilson (default) or Clopper–Pearson CIs, LR+ and LR-, Youden J (first maximum), closest to (0,1). AUC by Mann–Whitney (ties 1/2); DeLong SE by midranks; Hanley–McNeil SE; CIs clipped to [0, 1]; z test of AUC = 0.5 with the DeLong SE. |
proportion_ci(x, n, level, method) | Wilson (1927); Clopper & Pearson (1934); Brown, Cai & DasGupta (2001) | Prism’s “Wilson/Brown” hybrid is not reproduced; plain Wilson is the oracle’s. |
delong_paired(a, b, labels, ci_level) | DeLong et al. (1988) | Two correlated AUCs; covariance of the structural components. No oracle fixture (identity tests only). |
agreement
| Function | Reference | Notes |
|---|---|---|
bland_altman(a, b, kind, ci_level) | Bland & Altman (1986), Lancet 327:307; (1999), Stat Methods Med Res 8:135 | Difference A - B, ratio A/B or 100 (A-B)/mean; bias, SD (n-1), limits bias +/- 1.96 SD (Prism’s constant at 95%; z quantile otherwise; exact-z limits also reported); bias CI with t(n-1); SE of a limit SD sqrt(1/n + 1.96^2/(2(n-1))) with t CIs; proportional bias by OLS of difference on mean. |
icc(rows, ci_level) | Shrout & Fleiss (1979), Psychol Bull 86:420; McGraw & Wong (1996), Psychol Methods 1:30 | ICC(1,1), (2,1), (3,1), (1,k), (2,k), (3,k) with F tests and F-based CIs (Satterthwaite df for ICC(2,.)), as R psych::ICC. Hand-tested on the Shrout–Fleiss example. |
cohen_kappa(table, weights, ci_level), cohen_kappa_ratings(r1, r2, weights, ci_level) | Cohen (1960); Fleiss, Cohen & Everitt (1969), Psychol Bull 72:323; Cicchetti & Allison (1971); Fleiss & Cohen (1973) | Unweighted, linear or quadratic weights; asymptotic SE (CI) and null SE (z test). Hand-tested. |
multivariate
| Function | Reference | Notes |
|---|---|---|
pca(rows, scaling) | Jolliffe (2002) ch. 1-3 | SVD of the centred (standardized by default, n-1 SD) matrix; eigenvalues S^2/(n-1), explained and cumulative ratios, eigenvectors (rows = components) flipped so the largest-magnitude entry is positive (first on ties), loadings eigenvector x sqrt(eigenvalue), scores U S, Kaiser count. sklearn PCA on the standardized matrix. |
parallel_analysis(fit, replicates, percentile, seed) | Horn (1965), Psychometrika 30:179; Glorfeld (1995) | Eigenvalues of seeded standard-normal data of the same shape and scaling; mean and percentile per component; components retained while above the percentile. |
covariance_matrix(rows) | n-1 denominator, compensated two-pass sums, complete rows. |
resample
| Function | Reference | Notes |
|---|---|---|
Xoshiro256::new(seed), next_u64, next_f64, below(n), normal, shuffle | Blackman & Vigna (2021), ACM TOMS 47:36; Steele, Lea & Flood (2014) SplitMix64; Lemire (2019) | xoshiro256** seeded by SplitMix64; 53-bit uniforms; unbiased bounded integers; normals by inversion; Fisher–Yates. Deterministic on every platform. |
bootstrap_ci(samples, stat, replicates, ci_level, method, seed) | Efron (1979); Efron (1987), JASA 82:171; Efron & Tibshirani (1993) ch. 14 | Each sample resampled within itself; percentile or BCa (z0 with ties counted 1/2, acceleration from the leave-one-out jackknife of every observation); quantiles of the replicates by type 7 interpolation; non-finite replicates dropped. |
bootstrap_mean_difference(a, b, replicates, ci_level, method, seed) | mean(a) - mean(b). | |
permutation_test(a, b, exact_limit, monte_carlo, seed) | Fisher (1935); Pitman (1937); Phipson & Smyth (2010) | Difference in means; exact enumeration of all C(n, n_a) splits when within exact_limit, else Monte Carlo with p = (count + 1)/(B + 1); two-sided and one-sided p. |
Results sheets
dioptra-core::compute turns each analysis specification into a results sheet on top of
the functions above. Every sheet has the same parts, in the order the UI shows them:
assumption checks, summary sentences, a methods line, typed sections, a checklist.
docs/ANALYSES.md describes each analysis kind.
Data. Excluded cells are never used; empty and text cells are missing. Paired and repeated-measures analyses align values by row and drop a row with a missing member, saying how many. Nested tables are averaged within each subgroup first (n counts the experimental units). Data sets entered as mean/SD/N (or mean/SEM/N, mean/CI/N) are used where the result depends only on the means, SDs and sizes: t tests (one-sample, Student, Welch), one-way ANOVA with its post tests, and two-way ANOVA cells. Dioptra builds a synthetic sample with exactly that mean, SD and N, so these results are exact; rank tests, paired designs, normality and outlier tests say they are not possible from summary data. Contingency counts must be non-negative whole numbers; the first bad cell is named and the table is left as it is. Survival codes must be 1 (event) or 0 (censored).
Assumption checks. Each check has a status (pass, caution, fail, not_checked),
a one-sentence meaning, and an alternative when the status is not pass. Every check
uses α = 0.05.
| Check | Rule | Used by |
|---|---|---|
| Normality | Shapiro–Wilk (3 ≤ n ≤ 5000), D’Agostino–Pearson as a second opinion (n ≥ 8). Pass when every p ≥ .05; caution when the smallest p is in [.01, .05) or only D’Agostino–Pearson objects; fail below .01; not checked below n = 3. Applied to each group (t tests, correlation), the paired differences or log ratios (paired tests), the residuals (ANOVA, regression, curve fits), the differences (Bland–Altman). | t tests, ANOVA, correlation (Pearson), regressions, Bland–Altman, outliers |
| Equal variances | Brown–Forsythe (Levene on medians) decides; Bartlett is shown. Caution below .05, fail below .01. Informational (pass) when the test in use does not assume them (Welch, Welch’s ANOVA). From summary data: the SD ratio (t test) or Bartlett (ANOVA). | t tests, one-way and two-way ANOVA, Mann–Whitney (as a shape check) |
| Sphericity | Mauchly’s W; pass when the Greenhouse–Geisser correction is on. Not checked for the two-way repeated design. | Repeated-measures ANOVA |
| Expected counts | Cochran’s rule: caution when more than 20% are below 5, fail when any is below 1; Fisher’s exact test offered. | Chi-square |
| Sample size | Fail below 2 per group; caution below a minimum per analysis (5 for t tests and ANOVA, 6 pairs for Wilcoxon, 10 for correlation and ROC, 30 pairs for Bland–Altman). | most |
| Linearity | Runs test (too few runs, exact Swed–Eisenhart p); replicates (lack-of-fit) test when X repeats. | Linear and nonlinear regression |
| Outliers | Grubbs at α = 0.05 on each group, as a note only (caution, never fail). Nonlinear fits use ROUT when asked. | t tests, ANOVA, correlation |
| Convergence | Caution when a parameter hit a constraint or is ambiguous; fail when the fit did not converge. | Nonlinear regression |
| Stated, not tested | Independence (every analysis), proportional hazards and uninformative censoring (survival), X without error (regression), representative samples (ROC), acceptable agreement (Bland–Altman). | — |
Summary sentences. APA style, estimate first: the effect with its CI, then the test
statistic with its degrees of freedom, then p in the project’s style (p_phrase), then an
effect size. Examples: “The mean difference (A − B) was 3.2 (95% CI 1.1 to 5.3),
t(17.4) = 3.21, p = .005, Hedges’ g = 1.40.”; “One-way ANOVA: F(2, 27) = 4.85, p = .016,
η² = .26 (ω² = .20).” plus one sentence per multiple-comparison family naming the
comparisons that differ. Estimates and CI limits share their decimals: two significant
digits of the CI width (at least two of the estimate), scientific notation below 0.001 or
from 10⁶. Test statistics have two decimals; fractional df one decimal (two below 10);
quantities bounded by 1 (r, η², R², AUC) drop the leading zero in APA style.
Methods line. Test, sidedness, correction or adjustment, CI level, data form (for example “computed from the entered mean, SD and N”) and the software version, ready to paste: “Welch’s unpaired t test, two-sided, 95% CI. Dioptra 0.2.0.”
Graphs. A sheet names a default graph family (graph_family) and carries runtime
graph material: the analysed data sets, significance brackets (adjusted p of each
comparison; all of them up to six, otherwise the significant ones), fit curves with
confidence or prediction bands evaluated at 200 X values (linear and, when X > 0, log
spaced), survival curves with bands and numbers at risk, ROC points, replacement series
(correlation pairs, PCA scores and scree, Bland–Altman ratio or percent differences).
Graphs
A graph in Dioptra is a specification, not a picture: which table or analysis it draws, which family (column, box, violin, scatter, Kaplan–Meier, and so on), how data map to marks, a style, and annotations. The plot engine turns the specification into a scene measured in points and writes it as SVG. What you see in the editor is that SVG; what you export is the same SVG, or a PDF or PNG made from it. There is no separate “export renderer” that can disagree with the screen.
This page is about using the editor. The graph reference lists every family with the data it needs and the options it reads, the themes and palettes in detail, and the specification format.
Making a graph
- From a table: Graph → New graph (Ctrl-Shift-G). The family defaults to the natural one for the table kind: scatter for XY, Kaplan–Meier for survival, SuperPlot for nested, column with points for the others.
- From a results sheet: Make graph. Brackets for the comparisons, fitted curves and their confidence bands come from the analysis and stay linked to it. Paired and repeated-measures analyses offer Graph with connecting lines: bars with a line per subject (Connecting lines and paired graphs).
A column graph shows every data point by default, with the error bar you choose (SD, SEM, 95% CI, range or IQR). A bar alone hides the distribution: two groups with the same mean and SD can look identical as bars and completely different as points. If you want bars without points, untick Points; it is your choice to make, not the default.
The editor
Click anything in the graph (an axis, its title, a tick label, a data series, the legend, a bracket) and it is outlined, and the inspector on the right shows its properties: for an axis, its title, scale (linear, log10 or log2), limits, tick interval, minor ticks, tick direction, number format, label angle and breaks; for a series, its colour, marker, size and line style. Edit a value and the graph redraws.
Drag the legend or an annotation to move it. Double-click text to edit it in place. Graph → Add places a significance bracket, a reference line or a text label. Every change is undoable.
Text in titles accepts a small markup: ^ for superscript and _ for subscript
(cm^2, CO_2), \alpha-style names or plain Unicode for Greek letters.
Size: millimetres and points
Journals specify figures in physical units, so Dioptra does too. Figure sizes are in millimetres (or inches); font sizes, line widths and marker sizes in points, where 1 pt is 1/72 inch. The size menu has presets for a single journal column (85 × 70 mm), a double column (174 × 100 mm) and a slide (120 × 90 mm); any other size is typed in. The default figure is 85 × 70 mm with 8 pt tick labels and 9 pt axis titles, which is legible when printed at that size.
Why not pixels? Because a pixel has no size until you print it. A figure designed at “800 pixels” will have 4 pt text at one resolution and 12 pt at another. A figure designed at 85 mm with 8 pt text has 8 pt text in the journal.
Themes
| Theme | For |
|---|---|
| Instrument (default) | Papers and posters. White, thin dark axes on two sides, ticks outside, no gridlines, Inter. |
| Night | Slides with a dark background. The same geometry, light lines and text. |
| Journal | Black-and-white print. Series differ by marker shape, fill and dash rather than colour; slightly smaller type. |
Font size, axis line width and marker size can be overridden for one graph without leaving the theme.
Palettes
The default palette is Okabe–Ito: eight colours chosen so that people with the common forms of colour blindness can still tell them apart. About one man in twelve has some colour-vision deficiency, so a figure that relies on red against green loses part of its audience. The other palettes are Paul Tol’s bright, muted and light sets, viridis and cividis for ordered data, Tol sunset for diverging data such as fold changes, and a grey ramp. Any series can be given its own colour.
Export
Export (Ctrl-Alt-E) offers three formats.
- PDF, with the fonts embedded as subsets. Opens identically everywhere and goes straight into a journal submission system or Illustrator, without missing-font substitutions.
- SVG, with real text (not outlines), grouped and named layers, and no clipping masks or invisible objects, so it edits cleanly in Illustrator or Inkscape.
- PNG at any resolution. The pixel size is the physical size times the resolution: 85 mm at 300 dpi is 85 / 25.4 × 300 = 1003.9, so 1004 pixels. The file records its resolution, so it imports at the right size into Word or PowerPoint.
The dialog states the exact file it will write before you write it. Copy graph as SVG puts the SVG on the clipboard.
Because the fonts are bundled and the engine does its own text measurement, the same graph produces the same bytes on Linux, Windows and macOS.
Layouts
A layout places several graphs on one page with panel letters (A, B, C…), for a multi-panel figure. Graph → New layout. Each panel stays a live graph; edit it and the layout follows. Export a layout exactly as a single graph.
Connecting lines and paired graphs
Why a line
Measure the blood pressure of eight people, give them a drug, measure again. Draw two bars of the mean, before and after, and the bars say the average fell by eight. They do not say whether everyone fell by eight, or whether two people fell by forty and the rest did not move. Those are different drugs.
The data know the difference, because each row is one person. A paired test uses that: it works on each person’s own change, and that is why a paired t test can find an effect that an unpaired test on the same numbers misses. A graph of paired data should show the same thing the test sees. The way to do it is to draw each person’s points and join them with a line, bar to bar. Parallel lines that all slope down are a consistent effect; lines that cross every which way are noise around a mean.
Dioptra draws these lines on any bar graph, dot plot, box plot or violin: connecting lines are an option of the graph, not a separate kind of graph.
A worked example
Paste eight subjects measured at baseline, week 4 and week 8 into a column table. The first row becomes the column titles. The third subject missed the week-4 visit, so that cell is empty.
Baseline Week 4 Week 8
131 124 118
118 116 105
142 127
109 104 101
125 117 112
137 126 119
114 112 104
128 119 116
Run Analyze → Repeated-measures one-way ANOVA (or a paired t test on two columns, a Wilcoxon test or a Friedman test). The results sheet’s toolbar button reads Graph with connecting lines: these analyses default to the paired graph. Click it.
You get bars of the mean with SEM error bars, every subject’s points on the bar centres, a thin grey line per subject, and the significance brackets from the analysis. Now open the Connecting lines section of the inspector:
- Off / Subjects / Mean / Median / Both chooses what is joined. Subjects joins the values of each row. Mean and Median join the summary of each bar instead. Both draws the subject lines and the line of the graph’s summary (the mean, or the median when Bars and points → Summary says median). In the picture above it is Both.
- Missing says what an empty cell does. Gap (the default) breaks that subject’s line there; Join across connects the values on either side. Subject 3 above has a baseline and a week-8 value only, so with Gap it draws no line at all: there is no pair of neighbouring visits to join. A line from baseline straight to week 8 would hide that the week-4 value was never measured; choose Join across when that is what you want.
- Subject lines: style (solid, dashed, dotted), width in points, opacity, and colour. Colour by is one colour for every line (Prism’s convention), the data set (each segment takes the colour of the bar it starts from), the subject (each row its own colour from a palette), or a grouping column (a column of the table that says, say, Male or Female for each row: lines take a colour per group, a legend appears, and that column is not drawn as a bar).
- Summary line: its own style, width and colour.
Everything is undoable, and the graph redraws as you change it. Click a subject line in the graph to select it: the inspector shows the same controls, and the status line names the row.
How lines are matched
Lines join the values in the same row, as Prism does. Row 5 in Baseline and row 5 in Week 8 are the same subject. If your data are in long format (one row per measurement), pivot them to wide first with Data → Transform… → Pivot long to wide.
Lines run in the order the categories appear on the axis, which is the order of the data sets in the table (or of the analysis’s chosen data sets). Reorder the columns to reorder the lines.
Lines meet the points
When points are jittered sideways so they do not overlap, each line ends at its own point’s jittered position, not at the bar centre, so lines and points always agree. With Bars and points → Jitter at 0 % the points sit on the centre line, the classic before–after look; the paired preset uses 0 %.
Every bar style
The same lines work whatever draws the categories. Switch the family in the toolbar and the connecting lines stay:
| Style | Family | Notes |
|---|---|---|
| Bars with points | column | the default for paired analyses |
| Bars only | column, Points off | subject lines still drawn, from the values |
| Dot plot (scatter) | strip | points with a mean or median line |
| Mean ± SD, SEM or 95% CI | strip, Points off | the summary tick and its error bar |
| Box and whiskers | box | lines start at the jittered points, or the centre without points |
| Violin | violin | lines join the points inside the violin |
Subject lines are drawn on top of bars, boxes and violins and beneath the points; error bars and the summary line sit above the points. Horizontal (Bars and points → Orientation) turns the graph so the categories run down the left; the lines turn with it. Log axes work as usual, and a line that crosses a broken axis is cut at the break, as the axis is.
Grouped tables
In a grouped table each row is a group (Day 1, Day 7, Day 14) and each data set has subcolumns. For repeated measures, subcolumn 1 of every data set is subject 1, subcolumn 2 is subject 2, and so on. Direction chooses which way the lines run:
- Within groups (default): inside each group, from one data set’s bar to the next (Saline to Drug on Day 1). This is Prism’s before–after on a grouped graph.
- Across groups: along each data set, from group to group (Saline on Day 1, Day 7, Day 14), Prism’s “each replicate”. The summary line then takes each data set’s colour.
Stacked bars draw no connecting lines: a stacked bar has no single height per data set to join.
In the file
A graph with connecting lines is an ordinary column (or strip, box, violin) graph with an
options.connecting block; the graph reference lists every field and the
file format chapter says how graphs of the old Before–after family
from Dioptra 0.2 are opened: as the dot plot that draws the same picture.
"options": {
"jitter": 0,
"connecting": { "connect": "both", "style": "solid", "colour_by": "single",
"opacity": 0.55, "missing": "gap" }
}
Graphs
How Dioptra draws figures: the graph specification the UI edits, the data each family
needs, the options each family reads, the themes and palettes, and the hit-test targets
the graph editor uses. The engine is crates/dioptra-plot; dioptra-core re-exports
its specification types, so the JSON below is exactly what create_graph,
update_graph, patch_graph and get_graph_spec carry.
The pipeline
dioptra_core::graph::extractreads the source table intoGraphData: series with X values (or category positions 1, 2, 3, …), Y values, raw replicates per X (missing cells kept as NaN so rows line up for paired graphs), precomputed error bounds for mean/SD/N-style tables, subgroup ids for nested tables, curves and at-risk tables.- The family turns spec + data into a plot: axis plans and marks in data units.
- The layout measures every label with the bundled Inter metrics, sizes the margins so tick labels, axis titles, legends, colour bars and tables never collide, maps marks to points, clips them geometrically (no clipping masks in exported files) and emits a scene in points.
- The scene becomes SVG (own writer), PNG (resvg, any DPI) or PDF (svg2pdf, fonts embedded as subsets). The SVG shown in the app is the SVG exported.
Sizes are in millimetres; strokes, ticks and fonts in points (1 pt = 1/72 in). The
default figure is 85 × 70 mm (a single journal column), 8 pt tick labels, 9 pt titles.
PNG pixel size is round(mm / 25.4 × dpi): 85 mm at 300 dpi is 1003.94 → 1004 px. PNGs
carry the DPI in a pHYs chunk.
The specification
Every field has a default, so {"source": "<table id>", "family": "column"} is a valid
spec, and merge patches can set one field at a time. Style fields that are absent mean
“the theme decides”.
{
"source": "<table or analysis id>",
"family": "column",
"title": "Optional heading",
"mapping": {
"datasets": [0, 2], // which datasets (empty: all)
"x": 0, "y": [1, 2], // flat columns, variables tables only
"label": 3, // flat column with point labels (volcano)
"error_bars": "sem", // none | sd | sem | ci95 | range | iqr
"points": true // data points over summaries
},
"style": {
"theme": "instrument", // instrument | night | journal
"palette": "okabe_ito", // absent: theme default
"size_mm": {"width": 85, "height": 70},
"font_pt": 8, // tick labels; titles are +1 pt
"line_pt": 0.75, // axis lines; data lines scale with it
"marker_pt": 4,
"transparent": false,
"datasets": [{"dataset": 1, "colour": "#D55E00", "marker": "square_open",
"marker_pt": 5, "line_style": "dashed", "line_pt": 1.2,
"label": "Knock-out"}]
},
"axes": {
"x": {"title": "Dose (nM)", "scale": "log10", "min": 0.1, "max": 1000,
"breaks": [{"from": 20, "to": 80}],
"ticks": {"major": 1, "minor": 1, "direction": "out", "length_pt": 3,
"format": "auto", "decimals": 1, "label_angle": 45},
"hidden": false},
"y": {"title": "Response (% of max)"},
"frame": "l", // l | box | none
"origin": "frame", // frame | zero
"grid": false,
"offset_pt": 0
},
"legend": {"position": "auto", "title": "Genotype", "at": {"x": 0.05, "y": 0.05}},
"options": { "...": "family options, below" },
"annotations": [
{"kind": "significance_bracket", "from": 0, "to": 2, "text": "p < 0.001"},
{"kind": "text_label", "text": "R^2 = 0.98", "x": 2, "y": 15, "size_pt": 8},
{"kind": "line", "x1": 0, "y1": 0, "x2": 10, "y2": 10, "dashed": true},
{"kind": "reference_line", "axis": "y", "value": 10, "label": "LLOQ", "dashed": true}
]
}
(Comments are for this document only; JSON has none.)
Notes:
- Text markup in titles, labels and category names:
^{…}and_{…}for super- and subscripts (^2,^-1and_itake one character or a signed number),\alpha…\omega,\Delta,\mu,\degree,\pm,\times,\le,\ge,\infty,\permilfor symbols (a following space or{}ends the name, as in TeX),\^,\_,\\for the literal characters. A newline in a category name forces a line break. - Ticks are placed by the extended Wilkinson algorithm on 1, 2, 2.5 and 5 × 10ⁿ
steps; the axis ends on a tick unless data sit on the last tick, in which case it runs
4% past it. Numbers never show floating noise; they switch to
m×10ⁿfor |x| ≥ 10⁶ or ≤ 10⁻⁴, and use a typographic minus. Log axes put majors on powers of ten (1, 10, 100 inside 0.001…10 000, 10ⁿ outside, or asformatasks), minors on 2…9 × 10ⁿ, and switch to 1–2–5 steps when the data span less than about a decade. Labels that would collide are thinned (the dropped ones stay as minor ticks); long category labels wrap onto up to three lines before they rotate to 45°. - Breaks cut the axis, bars and gridlines and draw a gap glyph.
- Legend position
autolets the family decide: right of the plot for most, inside top right for survival, inside bottom right for ROC, above a volcano plot.atplaces the legend’s top-left corner at fractions of the plot area (set by dragging). - Significance brackets run between 0-based categories; with no
ythey sit above the highest point, bar or error bar in their span and stack when they would overlap. - Markers:
circle,square,triangle,triangle_down,diamond,plus,cross,none, and_openvariants (circle_open). Line styles:solid,dashed,dotted,dash_dot,none.
Family options (options)
| Field | Default | Used by |
|---|---|---|
summary | mean | column, strip, box (both lines), superplot (mean or median) |
arrangement | grouped | column on grouped tables (grouped, stacked, percent) |
connect | false | scatter: join points |
notch | false | box |
violin_inner | box | violin (box, strip, quartiles, none) |
truncate | true | violin, raincloud: cut the density at the data range |
bandwidth | Silverman | violin, raincloud, density |
bin_width, bins | Freedman–Diaconis | histogram |
density | false | histogram: density instead of counts |
jitter | 0.5 | column, box, strip, raincloud, estimation, superplot |
mean_line | true | the retired before_after family (read when a 1.0 file is migrated) |
bootstrap, seed | 5000, 20261011 | estimation (and the jitter phase) |
cluster_rows, cluster_columns | false | heatmap |
cell_values | false | heatmap |
ci_band, censor_ticks, at_risk | true | kaplan_meier |
fc_threshold, p_threshold, label_top | 1, 0.05, 5 | volcano |
reference | 0 (1 on log axes) | forest |
donut_hole | 0.55 | donut |
connecting | off | column, strip, box, violin: connecting lines (below) |
orientation | vertical | column, strip, box, violin (vertical, horizontal; axes.y stays the value axis) |
error_direction | both | column, strip (both, above, below) |
cap_pt | family default (5 or 6) | column, strip: error-bar cap width in points, 0 for none |
bar_width | family default | column, strip (summary line), box, violin: width as a fraction of the slot |
group_width | 0.84 | column, strip, box, violin: slot width as a fraction of the category spacing |
bar_fill | tint | column (tint, solid, empty) |
summary_line | true | strip: draw the mean or median line |
Per data set, style.datasets[].pattern draws a fill pattern over bars: hatch,
back_hatch, crosshatch, horizontal, vertical, dots. Patterns are clipped strokes,
so SVG, PDF and PNG carry the same geometry.
Connecting lines (options.connecting)
| Field | Default | Meaning |
|---|---|---|
connect | none | individuals (a line per row), means, medians, both (subjects and the graph’s summary) |
style | solid | subject lines: solid, dashed, dotted, dash_dot, none |
width_pt | theme axis width | subject line width |
colour_by | single | single (colour, or the axis colour), dataset (each segment the colour of the bar it leaves), subject (each row from palette), group (rows coloured by the values of table data set group_column, with a legend; that column is not plotted) |
opacity | 0.55 | subject line opacity |
missing | gap | gap: a missing value breaks the line; skip: join the values either side |
direction | within_groups | grouped tables: subcolumn k of each data set in a group is subject k (within_groups), or each data set’s subcolumn k is joined across the groups (across_groups, Prism’s “each replicate”) |
summary_style, summary_width_pt, summary_colour | solid, 2 × data width, axis colour | the mean or median line (on across_groups, the data set’s colour) |
Subjects are matched by row, as in Prism. Lines attach to the jittered point positions, so
lines and points always meet; with jitter: 0 the points sit on the category centre (the
aligned before–after look). Drawing order: bars, boxes or violins; subject lines; points and
error bars; the summary line. Lines are cut where they cross an axis break. Stacked bars draw
no lines. Each subject line is an editor target (subject-<row>, subject-g<group>-<row> or
subject-s<data set>-<row>) pointing at /options/connecting; the summary line is
connect-summary.
Families
Each entry says what data it draws, what it computes, and gives a minimal spec. Every family honours the theme, palette, per-dataset styles, sizes, axis settings, legends and annotations.
scatter
XY points. Data: XY tables (X column, Y datasets with replicates or mean/SD/N formats) or
variables tables with mapping.x/mapping.y. Replicates are drawn as their mean (or
median) with error bars as error_bars asks; with error_bars: "none" and points
the raw replicates are drawn faintly. X error bars come from the data. Fit curves and
CI bands in GraphData.curves are drawn under the points in the series colour.
{"source": "t1", "family": "scatter", "mapping": {"error_bars": "sem"}}
line
As scatter with every series joined by a line in its line style, markers on.
{"source": "t1", "family": "line", "style": {"datasets": [{"dataset": 0, "line_style": "dashed"}]}}
column
Bars of the mean or median with error bars and the points on top (default). Column
tables give one bar per dataset; grouped tables give clusters by row with a legend;
arrangement stacks them or scales stacks to 100%. On log Y axes bars start at the
axis. Dark bars get upward error bars only. connecting joins each row’s points across
the bars and/or the means; orientation: horizontal lays the bars sideways.
{"source": "t2", "family": "column", "mapping": {"error_bars": "sd", "points": true}}
box
Tukey box plots: quartiles (type 7), whiskers to the furthest point within 1.5 IQR,
outliers as open points; notch draws median ± 1.58 IQR/√n; points shows every value.
{"source": "t2", "family": "box", "options": {"notch": true}}
violin
Gaussian kernel densities mirrored about each category, cut at the data range unless
truncate is false, with a slim box (default), the points, or quartile lines inside.
{"source": "t2", "family": "violin", "options": {"violin_inner": "quartiles"}}
strip
All points, jittered deterministically (density-scaled, golden-ratio sequence), with the
mean or median line and error bars.
{"source": "t2", "family": "strip", "mapping": {"error_bars": "sd"}}
raincloud
Half violin (cloud), slim box and jittered points (rain) per category.
{"source": "t2", "family": "raincloud"}
before_after (retired)
Format 1.0’s paired family. A spec naming it is drawn, and on open migrated, as the strip
spec with the same picture: jitter: 0, no error bars, no summary tick,
connecting.connect: individuals (or both when mean_line was on). New graphs use the
paired preset (column bars with subject lines) or the before_after preset, both of which
create_graph accepts as family names.
{"source": "t3", "family": "column", "options": {"jitter": 0, "connecting": {"connect": "individuals"}}}
estimation
Gardner–Altman: raw data with mean ± SD as a gapped line beside each group; on the
right, a floating axis anchored at the control mean (the first dataset) with each
group’s mean difference, its bootstrap distribution and its bias-corrected and
accelerated 95% CI (5000 resamples, fixed seed, so the figure is reproducible).
{"source": "t3", "family": "estimation", "options": {"bootstrap": 5000, "seed": 20261011}}
superplot
Nested tables: small points for every observation coloured (journal: shaped) by
subgroup, a large point per subgroup mean, and the mean ± error of the subgroup means,
so the error bar reflects biological replicates, not cells.
{"source": "nested1", "family": "superplot", "mapping": {"error_bars": "sem"}}
histogram
Shared bins (Freedman–Diaconis width rounded to 1, 2 or 5 × 10ⁿ, or bin_width/bins),
counts or density, translucent when several datasets overlap.
{"source": "t2", "family": "histogram", "options": {"bin_width": 0.5}}
density
Kernel density curves with a light fill, one per dataset.
{"source": "t2", "family": "density"}
heatmap
Rows are table rows (row titles), columns are datasets; cells coloured on viridis
(sequential) or Tol sunset (diverging, symmetric around 0 when the data cross it), with
a colour bar. Rows and columns can be reordered by average-linkage clustering on
Euclidean distance with dendrograms; cell_values prints the numbers.
{"source": "g1", "family": "heatmap", "options": {"cluster_rows": true, "cluster_columns": true}}
correlation_matrix
Pearson correlations between datasets (variables, paired by row), lower triangle,
coefficients printed, diverging scale from −1 to 1, square cells.
{"source": "vars1", "family": "correlation_matrix"}
kaplan_meier
Survival tables (X = time, 1 = event, 0 = censored): step curves in percent, censor
ticks, 95% CI bands (exponential Greenwood, log(−log) scale) and the number at risk under
each X tick. Curves supplied by an analysis are drawn instead when present.
{"source": "surv1", "family": "kaplan_meier", "options": {"ci_band": false}}
roc
The first dataset holds controls, each later one a patient group (higher values mean
disease): empirical ROC curves in percent, the chance diagonal, AUC in the legend.
{"source": "t3", "family": "roc"}
bland_altman
The first two datasets paired by row: difference against mean, bias and 95% limits of
agreement (bias ± 1.96 SD) with their 95% CIs shaded and labelled.
{"source": "t3", "family": "bland_altman"}
volcano
Variables table with mapping.x = log₂ fold change and mapping.y = p values (or
−log₁₀ p): threshold lines, up/down/not significant colouring with counts in the
legend, and labels (from mapping.label or row titles) for the label_top most
significant points, placed to avoid each other.
{"source": "rnaseq", "family": "volcano", "mapping": {"x": 0, "y": [1], "label": 2}}
forest
One row per estimate, top to bottom, with its CI, a reference line and the numbers in a
column on the right. Data: a dataset in a lower_upper, mean_ci_n or mean/SD format
(the error bounds), or three datasets read as estimate, lower, upper; row titles label
the rows. Log X axes suit ratios.
{"source": "or1", "family": "forest", "axes": {"x": {"scale": "log10"}}}
pie, donut
Parts of a whole from a parts table (or one value per dataset), clockwise from twelve
o’clock, percentages in slices of 5% or more, shares in the legend. Slice k takes its
colour from style.datasets entries with dataset: k.
{"source": "parts1", "family": "donut", "options": {"donut_hole": 0.6}}
Sample quantiles against normal quantiles (Blom positions) with a line through the
quartiles, per dataset.
{"source": "t2", "family": "qq"}
residuals
Observed minus fitted against X: the fitted curve of the series from the analysis when
supplied, otherwise a least-squares line; a dashed zero line.
{"source": "t1", "family": "residuals"}
Themes
| Theme | Background | Axes | Type | Lines | Notes |
|---|---|---|---|---|---|
instrument (default) | white | #222, 0.75 pt, L frame, ticks out 3 pt | Inter 8 pt ticks, 9 pt titles, 10 pt heading | data 1 pt, markers 4 pt | no gridlines; bars and boxes filled with a 70% tint of the series colour |
night | #16191a | #c9cdca | light text #e8eae8 | as instrument | for slides; the first Okabe–Ito colour becomes light grey |
journal | white | black, 0.5 pt | Inter 7 pt / 8 pt (Inter runs wider than Helvetica, so a journal that substitutes Helvetica still fits) | data 0.75 pt, markers 3.5 pt | black and white: series differ by marker shape and fill (●○■□▲△◆◇) and dash; heatmaps keep a colour-blind-safe scale (set palette: "grey" for grey) |
font_pt, line_pt and marker_pt override the theme; titles keep their offset from
font_pt, data lines scale with line_pt.
Palettes
| Name | Kind | Colours |
|---|---|---|
okabe_ito (default) | categorical | #000000 #E69F00 #56B4E9 #009E73 #0072B2 #D55E00 #CC79A7 #F0E442 (yellow last for white paper) |
tol_bright | categorical | #4477AA #EE6677 #228833 #CCBB44 #66CCEE #AA3377 #BBBBBB |
tol_muted | categorical | #332288 #88CCEE #44AA99 #117733 #999933 #DDCC77 #CC6677 #882255 #AA4499 |
tol_light | categorical | #77AADD #EE8866 #EEDD88 #FFAABB #99DDFF #44BB99 #BBCC33 #AAAA00 #DDDDDD |
viridis | sequential | 11 stops #440154 … #FDE725; categorical use samples 0–0.85 |
cividis | sequential | 11 stops #00224E … #FEE838 |
tol_sunset | diverging | #364B9A … #EAECCC … #A50026 (11 stops) |
grey | categorical / sequential | #000000 #666666 #999999 #BBBBBB #333333; ramp #F4F4F4 → #111111 |
Colours are tokens: a per-dataset colour overrides the palette.
Hit-test targets (graph editor)
render_graph and render_layout answer
{"type": "svg", "svg": "<svg …>", "width_mm": 85, "height_mm": 70,
"targets": [{"id": "axis-x-title", "target": "/axes/x/title",
"bbox": {"x": 92.4, "y": 182.1, "width": 54.3, "height": 8.7}}]}
bbox is in points in the SVG’s viewBox coordinates (the SVG is width_mm wide;
multiply by the displayed pixels per point to draw a selection box). Every <g> with an
id also carries data-target with the same pointer, so a click on any SVG element can
walk up to its group. Targets, in drawing order:
| id | target | what |
|---|---|---|
plot-area | /mapping | all data marks |
series-{i} | /style/datasets/{d} | one plotted series; d is its dataset index (find or create the style.datasets entry with dataset: d) |
curve-{i}, pairs, summary, cells, reference, thresholds, labels, difference-{j}, agreement | /mapping or /options/… | family layers |
annotation-{i} | /annotations/{i} | a bracket, label or line |
grid | /axes/grid | gridlines |
axis-x, axis-y | /axes/x, /axes/y | axis line, ticks and labels |
axis-x-labels, axis-y-labels | /axes/x/ticks, /axes/y/ticks | tick labels |
axis-x-title, axis-y-title | /axes/x/title, /axes/y/title | axis titles |
axis-y2, axis-y2-title | /axes/y, /axes/y/title | estimation difference axis |
axis-x-break, axis-y-break | /axes/x/breaks, /axes/y/breaks | gap glyphs |
frame | /axes/frame | box frame |
colour-bar | /style/palette | heatmap scale |
dendrogram-rows, dendrogram-columns | /options/cluster_rows, /options/cluster_columns | |
estimates | /options | forest number column |
at-risk | /options/at_risk | survival table |
legend, legend-item-{i} | /legend, /style/datasets/{d} | legend and its entries |
title | /title | heading |
In layouts, panel i is panel-{i} → /graphs/{i} and every id inside it is prefixed
p{i}- (p1-axis-x).
Layout style
update_layout takes an optional style (absent: unchanged), and get_layout returns it:
{"width_mm": 183, "font_pt": 6, "line_pt": 0.5, "letters": "lower", "letter_pt": 8}
width_mm resizes every panel, keeping its aspect ratio, so the columns and their letter
gutters fill exactly that width; font_pt and line_pt replace each graph’s own in every
panel; letters is upper (default), lower or none, set bold at letter_pt (default
10). Fonts and lines are never scaled with the panel. Setting anything but the default needs
Dioptra Lab; files keep drawing as saved in every build.
Determinism
The same spec and data give the same bytes on every platform and in the browser:
fonts are bundled (Inter Regular, SemiBold, Bold, Italic) and measured from their own
tables, PNG and PDF never consult system fonts, jitter and bootstrap use a fixed-seed
SplitMix64 generator, and no output depends on hash-map order. Golden SVGs for every
family are in crates/dioptra-plot/tests/snapshots/; regenerate them with
UPDATE_SNAPSHOTS=1 cargo test -p dioptra-plot. Series with more than 20,000 points are
decimated to one point per 0.75 pt cell (a pixel at 96 dpi) at render time, and the SVG
says so in a comment.
Journal figures
Dioptra Lab. SVG, PDF and PNG export at any size, multi-panel layouts and colour-blind-safe palettes are free.
Journal presets
Journal in the graph editor or layout toolbar. Pick a journal and a column width, then Apply to this graph or Apply to every figure (every graph and layout in the project). One undo takes the whole change back.
A preset sets the width, scales the height with it (up to the journal’s maximum), sets the type size and line width, and switches the slides theme to the light one. On a layout it sets the page width, one type size and line width for every panel, and the journal’s panel letters.
Type: tick labels at 7 pt, raised where the smallest text the engine draws (85% of the tick labels: point labels, numbers at risk) would fall below the journal minimum, and lowered to keep axis titles (+1 pt) under the maximum. Axis lines at twice the journal’s minimum line width, at least 0.5 pt; data lines and outlines follow.
| Journal | Single | 1.5 | Double | Max height | Text | Lines | Raster (line / photo / combination) | Files | Letters |
|---|---|---|---|---|---|---|---|---|---|
| Nature | 89 mm | 120 mm | 183 mm | 170 mm | 5–7 pt, Helvetica or Arial | 0.25 pt | 1000 / 300 / 450 dpi | PDF, EPS, AI | a b c, 8 pt |
| Cell Press | 85 mm | 114 mm | 174 mm | 225 mm | 6–8 pt, Arial | 0.25 pt | 1000 / 300 / 500 dpi | TIFF (LZW), PDF, EPS, AI | A B C, 8 pt |
| Science | 57 mm | 121 mm | 184 mm | 230 mm | ≥ 6 pt, Helvetica | 0.28 pt | 1200 / 300 / 600 dpi | PDF, EPS, AI, TIFF | A B C, 9 pt |
| PLOS | 132 mm | — | 190.5 mm | 222.3 mm | 8–12 pt, Arial | 0.25 pt | 300 dpi | TIFF (LZW), EPS | A B C, 10 pt |
| eLife | 100 mm | — | 200 mm | — | 6–12 pt, sans-serif | 0.25 pt | 300 dpi | TIFF, PDF, EPS, AI, JPG | A B C, 10 pt |
| ACS | 84.7 mm | — | 177.8 mm | 232.8 mm | ≥ 4.5 pt, Helvetica or Arial | 0.5 pt | 1200 / 600 / 600 dpi | TIFF, PDF, EPS | A B C, 8 pt |
| Elsevier | 90 mm | 140 mm | 190 mm | — | ≥ 6 pt, Arial or Helvetica | 0.1 pt | 1000 / 300 / 500 dpi | EPS, PDF, TIFF, JPEG | A B C, 8 pt |
| APS (Physical Review) | 86 mm | 130 mm | 178 mm | 237 mm | ≥ 8 pt | 0.5 pt | 600 dpi | EPS, PS, PNG, JPG | a b c, 8 pt |
| The Journal of Physiology | 85 mm | — | 170 mm | 230 mm | ≥ 8 pt, sans-serif | 0.25 pt | 600 dpi | PDF, EPS, TIFF | A B C, 10 pt |
The figure check never accepts a line thinner than 0.25 pt, whatever the journal.
Sources, read on 14 October 2026:
- Nature: https://research-figure-guide.nature.com/figures/building-and-exporting-figure-panels/, https://www.nature.com/documents/nature-final-artwork.pdf, https://research-figure-guide.nature.com/figures/extended-data-formatting-guidelines/
- Cell Press: https://www.cell.com/figureguidelines
- Science: https://www.science.org/content/page/instructions-preparing-revised-manuscript
- PLOS: https://journals.plos.org/plosone/s/figures
- eLife: https://elife-rp.msubmit.net/html/elife-rp_author_instructions.html
- ACS: https://researcher-resources.acs.org/publish/author_guidelines?coden=jacsat
- Elsevier: https://www.elsevier.com/about/policies-and-standards/author/artwork-and-media-instructions/artwork-sizing, https://www.elsevier.com/about/policies-and-standards/author/artwork-and-media-instructions/artwork-types
- APS: https://journals.aps.org/authors/style-basics
- The Journal of Physiology: https://jp.msubmit.net/cgi-bin/main.plex?form_type=display_requirements
Journals revise these pages; the author guidelines of the journal you submit to decide.
Figure check
The Journal popover checks the figure against the chosen journal each time it changes. It reads the drawing that is exported, so it checks what the journal receives.
| Rule | Flags |
|---|---|
| Text size | Text below the journal minimum, super- and subscripts below it, text above the maximum. A figure wider than the journal’s full width is checked at the size the journal reduces it to. |
| Line width | Lines thinner than the journal minimum or 0.25 pt. Strokes in the background colour (gaps between slices or cells) are not lines. |
| Resolution | PNG and TIFF below the journal’s resolution: line art for black-and-white figures, combination for colour. |
| Colour vision | Two data-set colours that differ for normal vision but not for a reader with deuteranopia, protanopia or tritanopia. |
| Panels | Panels of a layout whose tick labels or axis lines differ in size. |
| Size | A figure wider than the journal’s full width or taller than its maximum; a width between its columns. |
Click an issue to select the element in the graph editor.
Colour vision is simulated with Machado, Oliveira and Fernandes (2009), doi:10.1109/TVCG.2009.113, at full severity on linear RGB. Two colours count as distinct when their CIEDE2000 difference (Sharma, Wu and Dalal 2005, doi:10.1002/col.20070) is at least 10. Neighbouring Okabe–Ito colours pass in all three simulations; a red and a green of similar lightness fail for deuteranopia.
Layouts at final size
Journal → Final size in the layout view:
- Letters: A B C, a b c or none; Letter size in points (bold).
- Page width: the panels are resized, keeping their shape, so the columns and their letter gutters fill exactly this width. Type and lines are not scaled.
- Text and Lines: one tick-label size and one axis line width in every panel, whatever each graph says.
Empty fields keep each graph’s own setting. A layout made this way draws the same in every copy of Dioptra; changing the settings needs Dioptra Lab.
TIFF
Export… → TIFF: 8-bit RGB with LZW compression, the resolution recorded in the file, rendered from the same drawing as PNG on a white background. EPS is not offered; PDF and SVG carry the same vectors.
Batch analysis
Part of Dioptra Lab. Importing a single file, plate-reader exports included, and replacing a table’s data are free.
A template is any project plus the table that receives the data. Batch analysis puts each data file into a copy of the template, recomputes every formula, analysis, graph and layout, and writes:
<file>.dioptra, the recomputed project;<file> - <graph>.pdf(or.png,.svg) for every graph and layout, at its set size;summary.csv, one row per file.
Files are read as by Replace table data: names and column formulas of the receiving table stay, the data change. A plate map, if given, applies to every plate-reader file.
The summary
The summary has a File column, one column per headline number and an Error column.
Headline numbers come from every section of every results sheet: the columns headed
Value, Best-fit or Estimate, and every column of p values. Each is named after the
analysis and the row, such as Fit · Data set A · LogEC50. A nonlinear fit gives its
parameters (Bottom, Top, LogEC50, HillSlope) and derived values (EC50); a t test gives t,
df, the P value, the difference and the effect sizes. Rename an analysis in the template to
name its columns.
A file that cannot be analysed (no data, a table of the wrong kind) gets its error in the summary; the others go on.
In the app
File → Batch analysis… asks for:
- the template: the open project, or a project file;
- the table that receives the data;
- the data files (on the desktop, files or a whole folder);
- the figure format and an optional plate map;
- the output folder (desktop). In the browser, everything downloads as one zip.
The dialog shows progress and, at the end, the summary and any errors.
File → Watch a folder… (desktop) applies a template to each new file that appears in a
folder, once its size has stopped changing for two seconds. Results go to the folder’s
analysed subfolder, with a summary.csv that grows by one row per file. Files already in
the folder are left alone. File → Stop watching the folder ends it; so does quitting.
From the command line
dioptra batch assay.dioptra --table "Plate" exports/ --out results --format png
Inputs are files or folders (their data files, not subfolders). --plate-map map.txt
applies a plate map; --dpi sets the PNG resolution (300). The exit status is non-zero
when any file fails.
The key is read from $DIOPTRA_LICENSE, or from the file the desktop app keeps:
| System | File |
|---|---|
| Linux | ~/.config/io.github.zahakj.dioptra/license.key |
| macOS | ~/Library/Application Support/io.github.zahakj.dioptra/license.key |
| Windows | %APPDATA%\io.github.zahakj.dioptra\license.key |
dioptra license shows it, dioptra license --set <key> stores it and
dioptra license --remove deletes it.
Methods and report
Dioptra Lab. The one-line methods text on each results sheet is free.
Methods paragraph
Methods paragraph… on a results sheet copies a paragraph for that analysis; File → Copy methods paragraph… covers every analysis in the project. For each analysis it names the data table, the test, sidedness, the correction for multiple comparisons and the post tests, the confidence level and the data form, then the software and version, and lists the references to cite once each. References are those of the methods Dioptra implements, as listed in Methods and references.
Plasma IL-6: Ordinary one-way ANOVA, Tukey's multiple comparisons (family-wise
α = 0.05). Analyses were performed in Dioptra 0.4.0 (https://github.com/ZahakJ/dioptra).
References: Fisher (1925); Tukey (1953); Kramer (1956), doi:10.2307/3001469.
Report
File → Export report… writes the project as one document:
- the title, the Dioptra version and the date;
- Data: each table with its first 20 rows;
- Methods: the methods paragraph;
- Results: each results sheet, with its summary sentences, methods line, assumption checks and tables, p values in the project’s style;
- Figures: every graph and layout, numbered, with its title as the caption.
| Format | Pages | Figures |
|---|---|---|
Word (.docx) | A4, 20 mm margins, Arial; headings use Word’s heading styles | PNG at 300 dpi, scaled to the text width |
| A4, 20 mm margins, Inter | Vectors with embedded fonts; graphs wider than the text are laid out again at its width |
Long tables continue on the next page with their header repeated.
Records for regulated labs
Part of Dioptra Lab. Dioptra supplies technical controls that support compliance with 21 CFR Part 11 and EU GMP Annex 11. Compliance also depends on the lab’s procedures, validation and storage.
The controls
Users and roles. Users are kept per computer, in lab-users.json in the app’s config
directory. Each has a user ID, a full name, a password and a role:
| Role | May |
|---|---|
| Analyst | change regulated projects, sign as author |
| Reviewer | also sign as reviewer or approver |
| Administrator | also add users, disable them and reset passwords |
Passwords are stored as PBKDF2-HMAC-SHA256 hashes (600,000 iterations, random 16-byte salt). Five failed sign-ins in a row lock an account until an administrator resets its password. Users are disabled, never deleted, so a user ID is never reused. Every creation, disabling, password reset and lockout is written to the users file’s own hash-chained access log. Dioptra refuses a users file whose accounts do not match its log.
The browser version keeps users in browser storage. It is not suitable for regulated work.
Audit trail. Records → Make this project regulated… turns on the audit trail of the
open project. It cannot be turned off. From then on every change (data, formulas,
exclusions, analysis options, graphs, layouts, deletions, undo and redo) needs a signed-in
user and a reason, and is appended to extensions/lab-audit.jsonl inside the project file:
| Field | Content |
|---|---|
seq | 1, 2, 3, … |
at | time, UTC |
user, name | user ID and full name |
action | what was done, in words |
command | the command, with large payloads summarised |
objects | ids of the tables, analyses, graphs and layouts affected |
reason | why |
version | record version |
content | SHA-256 of the project content after the change |
prev | hash of the previous entry |
hash | SHA-256 of the canonical JSON of the entry without hash |
Opening a regulated project checks the chain. A changed, missing or reordered entry, or project content that differs from the last entry, is reported at once.
Electronic signatures. Records → Sign… asks for the password again, then records the signer’s name and ID, the date and time, the meaning (authored, reviewed or approved), the SHA-256 of the project content and the hash of the trail head, as an entry of the trail. A signed project is locked. Records → New version… records a reason and starts the next version; the earlier signatures stay in the trail, marked superseded. The audit-trail viewer and the audit report show each signature with its name, date and time, meaning and status: valid, superseded, or content changed since signing.
Verification. Records → Audit trail… lists the entries, filters by user and object, and verifies the chain. Records → Export audit report… writes it as PDF or CSV. Verifying, viewing and exporting need no license, and a signed record stays locked when a license ends. Setting up users, making a project regulated, signing and starting a new version need Dioptra Lab.
What the controls cover
| Requirement | Covered by |
|---|---|
| 11.10(a) validation | customer procedure |
| 11.10(b) accurate and complete copies | project file (open JSON), audit report (PDF, CSV) |
| 11.10(c) protection and retrieval of records | customer procedure (storage, backup, retention) |
| 11.10(d) limited system access | users and passwords; access to the computer and files is a customer procedure |
| 11.10(e) audit trail | audit trail: who, when, what, why; append-only, hash-chained; retention is a customer procedure |
| 11.10(f) operational checks | reason and sign-in required for every change; signed records locked until a new version |
| 11.10(g) authority checks | roles: only administrators manage users, only reviewers and administrators sign as reviewer or approver |
| 11.10(h) device checks | customer procedure |
| 11.10(i) training | customer procedure |
| 11.10(j) accountability for signatures | customer procedure |
| 11.10(k) systems documentation | customer procedure; this manual |
| 11.50 signature manifestation | name, date and time, meaning, in the viewer and the report |
| 11.70 signature and record linking | signature bound to the content hash and the trail head |
| 11.200(a)(1) two components | user ID and password; the password is asked again for every signature |
| 11.200(a)(2), (a)(3) genuine owner only | lockout after five failures; customer procedure |
| 11.300(a) unique IDs | IDs never reused; users disabled, not deleted |
| 11.300(b) periodic password revision | customer procedure; administrators reset passwords |
| 11.300(c) lost credentials | administrator reset, recorded in the access log |
| 11.300(d) unauthorised use | lockout, recorded in the access log; reporting is a customer procedure |
| 11.300(e) device testing | customer procedure |
| Annex 11 §9 audit trails | audit trail with reasons; PDF and CSV report; review is a customer procedure |
| Annex 11 §12.1, §12.2 access control | users and roles; physical and network security are customer procedures |
| Annex 11 §12.3 access changes recorded | access log of the users file |
| Annex 11 §12.4 identity, date and time of changes | audit trail |
| Annex 11 §14 electronic signatures | signatures in the trail: permanently linked to the record, with date and time |
Limits of a desktop system
Dioptra runs on the lab’s own computers without a server, as a closed system in the sense of 11.10. The project file and the users file are ordinary files. Anyone who can write them, such as an operating-system administrator, can delete or replace them, including removing the trail from a project. The hash chain uses no secret key: it makes an edit evident, not impossible, and someone who rewrites the whole chain consistently is not detected. Keep regulated projects and the users file on storage with access control, backups and a retention policy, and restrict administrator rights on the computers.
Validation (IQ, OQ, PQ) and the standard operating procedures are the lab’s responsibility. The validation pack supplies the compliance matrix, traceability, IQ and OQ protocols and the automated OQ (Records → Run OQ…).
Validation pack
The validation pack for Dioptra Lab is kept with the source, for QA readers: docs/validation. It holds the Part 11 and Annex 11 compliance matrix, requirements traceability, IQ and OQ protocols, PQ guidance, change control and the supplier assessment. The automated OQ runs from Records → Run OQ….
The .dioptra project file
A .dioptra file is a zip archive (deflate) of pretty-printed JSON. unzip -l lists it;
unzip extracts it; git diff on the extracted tree shows exactly what changed. Keys are
written in a stable order; numbers use shortest round-trip representation; empty cells are
null.
manifest.json format version, Dioptra version, created/modified timestamps
project.json title, p-value style, display order of objects, layouts
tables/<id>.json one table: kind, datasets, subcolumns, columns (columnar arrays),
row titles, column sources and formulas, excluded cells, text cells,
edit log
analyses/<id>.json analysis spec (table id, kind, options); no results
graphs/<id>.json graph spec (source id, family, mapping, style, axes, legend, options, annotations; docs/GRAPHS.md)
results/<id>.json cached results sheets (optional; recomputed when absent or stale)
history.jsonl command log: one JSON object per line, newest last
extensions/<name> UTF-8 files kept for other components, e.g. Dioptra Lab's audit
trail; read back and saved unchanged by the core
manifest.json
{ "format": "dioptra/1.2", "app": "0.4.0", "created": "2026-10-11T09:00:00Z", "modified": "..." }
Readers must accept any dioptra/1.x; unknown keys are preserved on round trip where
possible and ignored otherwise.
tables/<id>.json
{
"id": "…", "title": "Dose response", "kind": "xy",
"x": { "title": "Dose (nM)", "values": [1, 10, 100, null], "source": {"kind": "pasted", "at": "…"} },
"row_titles": ["", "", "", ""],
"datasets": [
{ "name": "Control", "format": {"kind": "values", "n": 3},
"subcolumns": [
{ "values": [0.1, 0.5, 0.9, null], "excluded": [3], "text": {"2": "n/a"},
"source": {"kind": "formula", "expr": "log10(A)"}, "decimals": 2,
"edits": [{"row": 1, "at": "…", "from": 0.4, "to": 0.5}] }
] }
],
"variables": null
}
valuesis a dense array the length of the table’s row count;nullmeans empty.excludedis a sorted list of row indices.textholds non-numeric entries by row index (kept, shown, never used in numerics).titleon a subcolumn names it (a replicate or a nested subgroup, e.g. from Prism’s subcolumn titles); absent when unnamed. The X column’s title is the same field.sourceis one ofentered,pasted {at},imported {file, sheet, column},formula {expr},derived {table, op}.editsis the sparse per-cell edit log that, withsource, gives each cell’s provenance.- For
variablestables, each dataset has a single subcolumn and atype. - Derived tables carry
derived_from: {table, op}at the top level.
history.jsonl
Each line is { "at": "…", "command": { … }, "label": "Paste 3×120 into Table 1" }. The
engine replays nothing from it on open; it is an audit trail and the source of the undo
timeline in the UI.
graphs/<id>.json
{
"id": "…", "title": "Graph of Paired t test of Table 1",
"spec": {
"source": "…", "family": "column",
"mapping": { "datasets": [0, 1], "error_bars": "sem", "points": true },
"style": { "theme": "instrument", "size_mm": { "width": 85, "height": 70 },
"datasets": [{ "dataset": 1, "pattern": "hatch" }] },
"axes": { … }, "legend": { "position": "auto" },
"options": { "jitter": 0, …,
"connecting": { "connect": "individuals", "style": "solid", "colour_by": "single",
"opacity": 0.55, "missing": "gap", "direction": "within_groups",
"summary_style": "solid" } },
"annotations": [{ "kind": "significance_bracket", "from": 0, "to": 1, "text": "p = .002" }]
}
}
Every field of the spec is listed in docs/GRAPHS.md. Fields at their defaults may be left
out; connecting, orientation, error_direction, bar_fill and summary_line are
written only when they differ from the default, so a graph without connecting lines is
written exactly as format 1.0 wrote it.
Versions and migrations
| Format | Dioptra | Change |
|---|---|---|
dioptra/1 | 0.1, 0.2 | First format |
dioptra/1.1 | 0.3 | Graph specs gain options.connecting, orientation, error_direction, cap_pt, bar_width, group_width, bar_fill, summary_line and style.datasets[].pattern. The before_after family is retired. |
dioptra/1.2 | 0.4 | Layouts gain style (page width, font size, line width, panel letters). Files may carry extensions/<name>. Readers of 1.1 open 1.2 files and ignore the additions. |
Migration 1.0 → 1.1 runs when a file is opened: a graph whose family is before_after
becomes the strip spec that draws the same picture (jitter: 0, error_bars: none,
summary_line: false, connecting.connect both when mean_line was on and individuals
otherwise, opacity 0.4). Nothing else changes; saving writes the current format. A 0.2 reader opens a
1.1 file (the minor number is additive) and ignores the new fields, so it draws such a graph
without its connecting lines.
Compatibility promise
The format version bumps the minor number for additive changes and the major number for anything a 1.x reader could misread. Readers of version N open any file ≤ N.
Correctness
A statistics program is a measuring instrument, and an instrument is only as good as its calibration. This chapter explains how Dioptra’s numbers are checked, then gives the full record: every comparison, every NIST data set, and the cases where a difference was investigated.
The short version: all 58 NIST StRD reference data sets used (univariate, linear regression, one-way ANOVA and nonlinear regression) pass, and 7,841 comparisons in 300 oracle cases across 28 families of methods agree with SciPy, statsmodels, lifelines and scikit-learn within the tolerance stored with each case, with no failures.
What “correct” means for a number
A computer stores a number in 64 bits (IEEE 754 double precision), which holds about 15 to 16 significant decimal digits. Every arithmetic step can lose some of them. Most of the time the loss is invisible. Sometimes it is catastrophic: subtract two nearly equal large numbers and the leading digits cancel, leaving mostly rounding noise.
A classic example is the variance. The textbook “one-pass” formula
s² = (Σx² − (Σx)²/n) / (n − 1)
is exact in algebra and dangerous in a computer. Take the values 100000001, 100000002, 100000003. The answer is 1. But Σx² is about 3 × 10¹⁶, beyond the range in which doubles hold every integer, and the subtraction of two such numbers leaves almost nothing of the right digits. A program using that formula can report a variance of 0, or a negative one. The two-pass formula (compute the mean, then sum squared deviations from it) gets 1 exactly. Both formulas are “correct”; only one is correct in a computer.
So the useful question is not “is the formula right?” but “how many digits of the answer are right?”
LRE: counting correct digits
The log relative error (LRE) answers that question with one number. If the certified value is c and the program reports x,
LRE = −log₁₀( |x − c| / |c| )
An LRE of 15 means the first 15 significant digits agree; 10 means 10 digits; 0 means none. If c = 1.2345678901 and x = 1.2345678312, the relative error is about 4.8 × 10⁻⁸ and the LRE is 7.3: seven digits right. (When c is 0, the absolute error is used instead.) McCullough (1998) introduced LRE for exactly this purpose: judging statistical software on problems with known answers. As a rough scale, 4 digits is enough for any figure or table in a paper, 7 is what a sound algorithm reaches on ordinary data, and 15 is all a double can hold.
LRE is capped at the number of digits the reference certifies: 15 for most NIST sets, 11 for nonlinear regression. A cap means “as right as can be checked”.
NIST StRD: certified answers to hard problems
The US National Institute of Standards and Technology publishes the Statistical Reference Datasets (StRD): data sets with answers certified to 15 digits by computations in multiple-precision arithmetic. They are graded by difficulty. The lower sets are ordinary data. The higher ones are built to break naive algorithms: values like 1000000000000.4 that differ only in the last digits (ANOVA SmLs07–09), a tenth-degree polynomial on nearly collinear predictors (Filip), nonlinear models whose parameters are almost redundant (MGH09, Bennett5).
Dioptra’s test suite reads each NIST file, runs the same routine the app uses, and computes the LRE of every certified quantity. The test fails if any falls below a set bar. The tables below show the result for every data set.
Some LREs are lower than 15 for a reason no program can beat. The certified value of a data set with entries like 1000000.1 is computed from the decimal numbers, but a computer cannot hold 1000000.1 in binary; it stores the nearest double, which is slightly different. The exact answer for the numbers the computer actually holds therefore differs from the certified answer in the ninth or tenth digit. To separate the algorithm’s error from this input rounding, the tables have starred columns: the LRE against the exact answer for the doubles, computed in rational arithmetic. There Dioptra reaches 15 digits, or close to it, nearly everywhere.
Oracle fixtures: agreeing with the reference implementations
NIST covers only four families of methods. For everything else (t tests, rank tests, the
studentised range, Fisher’s exact test, Kaplan–Meier, ROC, PCA and so on) the reference is
the established open-source implementation: SciPy, statsmodels, lifelines and
scikit-learn. A Python script in the repository (scripts/oracle/) runs those libraries on
fixed inputs and writes the inputs and outputs to JSON fixture files with a tolerance for
each case. Dioptra’s tests read the fixtures and compare.
The tolerance is chosen per quantity: tight (10⁻¹² relative) for exact rational results such as exact p values, looser where the reference itself is approximate (SciPy’s randomised Dunnett integration, for example). Where Dioptra and a reference disagree beyond rounding, the disagreement was investigated to its source, often in high-precision arithmetic. Several of the notes below record cases where the reference library, not Dioptra, was the less accurate of the two.
A method with no reference check does not ship. The few methods without an oracle fixture (intraclass correlation, Cohen’s kappa, the bootstrap, permutation tests) are tested against published worked examples and mathematical identities, listed at the end of the record.
Reproducing the record
Everything below is regenerated by
cargo run -p dioptra-stats --example correctness_report
and the same comparisons run in cargo test -p dioptra-stats on every change.
Generated from cargo run -p dioptra-stats --example correctness_report (the same
comparisons as cargo test -p dioptra-stats), then annotated. This page covers special
functions, distributions, descriptive statistics, t tests, Mann–Whitney, Wilcoxon,
normality tests, outliers, ANOVA (one-way, two-way, repeated measures, Kruskal–Wallis,
Friedman), multiple comparisons, contingency tables, correlation, linear regression,
nonlinear regression (curve fitting), survival analysis, ROC curves, Bland–Altman and PCA.
Intraclass correlation, Cohen’s kappa, the bootstrap and permutation tests have no oracle
fixture; they are tested against published worked examples and identities (last section).
Every comparison uses the tolerance stored with its fixture case
(research/05-nist-and-oracle-notes.md §3); exact discrete p-values (Mann–Whitney,
Wilcoxon) are tightened to 1e-12 relative as that policy asks.
Oracle fixtures
| Family | Cases | Comparisons | Failures | Worst error / tolerance | Max rel. error | Min digits |
|---|---|---|---|---|---|---|
| incomplete beta + inverse | 10 | 315 | 0 | 4.5e-1 | 4.5e-13 | 12.3 |
| incomplete gamma + inverse, lgamma, digamma | 8 | 184 | 0 | 1.7e-1 | 1.7e-13 | 12.8 |
| normal | 1 | 82 | 0 | 4.0e-3 | 4.0e-16 | 15.4 |
| t, F, chi-square | 27 | 1086 | 0 | 1.5e-1 | 1.5e-13 | 12.8 |
| binomial, hypergeometric | 9 | 177 | 0 | 4.9e-2 | 7.3e-15 | 14.1 |
| noncentral t | 6 | 66 | 0 | 7.8e-3 | 7.8e-13 | 12.1 |
| studentized range | 90 | 855 | 0 | 5.1e-2 | 8.4e-6 | 5.1 |
| Dunnett distribution | 2 | 12 | 0 | 2.7e-5 | 2.0e-9 | 8.7 |
| descriptive | 12 | 385 | 0 | 1.8e-1 | 2.4e-13 | 12.6 |
| t tests | 21 | 235 | 0 | 9.5e-5 | 3.6e-12 | 11.4 |
| Mann-Whitney | 7 | 96 | 0 | 2.2e-4 | 1.4e-15 | 14.8 |
| Wilcoxon signed-rank | 6 | 165 | 0 | 1.6e-7 | 2.2e-15 | 14.7 |
| normality | 8 | 180 | 0 | 1.1e-5 | 1.3e-13 | 12.9 |
| outliers (Grubbs, ROUT) | 5 | 207 | 0 | 2.9e-1 | 2.9e-9 | 8.5 |
| one-way ANOVA (+ Welch, Brown-Forsythe, Bartlett, Levene) | 5 | 194 | 0 | 3.5e-7 | 5.1e-15 | 14.3 |
| two-way ANOVA (types I, II, III) | 2 | 140 | 0 | 4.9e-6 | 4.9e-14 | 13.3 |
| repeated-measures ANOVA, sphericity, Friedman | 3 | 81 | 0 | 3.9e-6 | 3.9e-14 | 13.4 |
| Kruskal-Wallis, Friedman, Dunn | 7 | 198 | 0 | 4.9e-6 | 4.9e-14 | 13.3 |
| Tukey-Kramer, Dunnett, p-value adjustment | 8 | 377 | 0 | 2.3e-4 | 2.0e-9 | 8.7 |
| sanity: statsmodels qsturng, SciPy QMC Dunnett | 8 | 48 | 0 | 1.9e-1 | 2.9e-1 | 0.5 |
| contingency (chi-square, Fisher, McNemar, trend) | 21 | 175 | 0 | 7.6e-6 | 7.6e-14 | 13.1 |
| correlation (Pearson, Spearman, Kendall) | 6 | 89 | 0 | 3.7e-7 | 1.8e0 | -0.3 |
| linear regression (simple, bands, runs, Deming, multiple) | 5 | 254 | 0 | 4.0e-3 | 8.7e-14 | 13.1 |
| nonlinear regression (fits, comparison, weighting, robust) | 12 | 498 | 0 | 3.8e-2 | 3.0e-4 | 3.5 |
| survival (KM, log-rank family, hazard ratios, Cox) | 3 | 685 | 0 | 4.7e-1 | 4.7e-6 | 5.3 |
| ROC (curve, Wilson CIs, AUC, DeLong, Hanley-McNeil, Youden) | 3 | 695 | 0 | 6.7e-5 | 1.0e0 | -0.0 |
| Bland-Altman (bias, limits, CIs, proportional bias) | 3 | 155 | 0 | 8.9e-4 | 1.1e-14 | 14.0 |
| PCA (eigenvalues, eigenvectors, loadings, scores) | 2 | 207 | 0 | 1.8e-4 | 1.8e-13 | 12.8 |
Worst error / tolerance: the largest |got - expected| / max(abs, rel |expected|) over the family, with each case’s own tolerance (< 1 passes). Max rel. error is over expected values with |value| > 1e-290, including tiny values where the abs tolerance governs; min digits = -log10(max rel. error).
NIST StRD univariate (LRE, McCullough 1998)
| Dataset | Difficulty | Mean | SD | Lag-1 r | Mean* | SD* | Lag-1 r* |
|---|---|---|---|---|---|---|---|
| PiDigits | Lower | 15.0 | 15.0 | 14.9 | 15.0 | 15.0 | 14.7 |
| Lottery | Lower | 15.0 | 15.0 | 15.0 | 15.0 | 15.0 | 15.0 |
| Lew | Lower | 15.0 | 15.0 | 14.8 | 15.0 | 15.0 | 15.0 |
| Mavro | Lower | 15.0 | 13.1 | 13.9 | 15.0 | 15.0 | 15.0 |
| Michelso | Lower | 15.0 | 13.8 | 13.4 | 15.0 | 15.0 | 15.0 |
| NumAcc1 | Lower | 15.0 | 15.0 | 15.0 | 15.0 | 15.0 | 15.0 |
| NumAcc2 | Average | 15.0 | 15.0 | 15.0 | 15.0 | 15.0 | 15.0 |
| NumAcc3 | Average | 15.0 | 9.5 | 12.2 | 15.0 | 15.0 | 15.0 |
| NumAcc4 | Higher | 15.0 | 8.3 | 11.0 | 15.0 | 15.0 | 15.0 |
LRE capped at 15 (NIST certifies 15 digits). Columns without a star score against the NIST certified values; starred columns against the exact statistics of the inputs as parsed into doubles (rational arithmetic), which separates the algorithm from input rounding: the NumAcc3/NumAcc4 values (1e6 + 0.1 and 1e7 + 0.1 steps) are not representable in binary, which caps any double-precision SD at 9.5 and 8.3 digits against the certified 0.1. Minimum LRE: 8.25 against certified values, 14.74 against the exact statistics of the double inputs.
NIST StRD linear regression (LRE)
| Dataset | Difficulty | Parameters | SDs | R^2 | Residual SD | ANOVA table | Min | Min* |
|---|---|---|---|---|---|---|---|---|
| Norris | Lower | 14.1 | 13.9 | 15.0 | 14.0 | 13.7 | 13.7 | 15.0 |
| Pontius | Lower | 13.5 | 13.8 | 15.0 | 13.8 | 13.5 | 13.5 | 15.0 |
| NoInt1 | Average | 14.7 | 15.0 | 15.0 | 15.0 | 14.7 | 14.7 | 15.0 |
| NoInt2 | Average | 15.0 | 14.9 | 15.0 | 15.0 | 14.5 | 14.5 | 15.0 |
| Filip | Higher | 14.0 | 14.8 | 15.0 | 14.8 | 14.4 | 14.0 | 15.0 |
| Longley | Higher | 14.6 | 14.9 | 15.0 | 15.0 | 14.8 | 14.6 | 15.0 |
| Wampler1 | Higher | 15.0 | 15.0 | 15.0 | 15.0 | 14.7 | 14.7 | 15.0 |
| Wampler2 | Higher | 13.2 | 15.0 | 15.0 | 15.0 | 14.6 | 13.2 | 15.0 |
| Wampler3 | Higher | 15.0 | 14.5 | 15.0 | 14.8 | 14.7 | 14.5 | 15.0 |
| Wampler4 | Higher | 15.0 | 14.5 | 15.0 | 14.8 | 14.7 | 14.5 | 15.0 |
| Wampler5 | Higher | 15.0 | 14.4 | 13.8 | 14.8 | 13.7 | 13.7 | 13.8 |
Parameters and SDs: minimum over the coefficients. ANOVA table: minimum over the regression and residual SS and MS and F (Wampler1’s F is infinite and not scored). Min*: minimum LRE against the exact least-squares solution for the inputs as parsed into doubles (rational arithmetic; values below 1e-12 scored by absolute error).
NIST StRD one-way ANOVA (LRE)
| Dataset | Difficulty | SS between | SS within | MS between | MS within | F | R^2 | Residual SD | Min | Min* |
|---|---|---|---|---|---|---|---|---|---|---|
| SiRstv | Lower | 14.0 | 13.1 | 14.0 | 13.1 | 13.1 | 13.2 | 13.4 | 13.1 | 15.0 |
| SmLs01 | Lower | 15.0 | 15.0 | 15.0 | 15.0 | 15.0 | 15.0 | 15.0 | 15.0 | 15.0 |
| SmLs02 | Lower | 15.0 | 15.0 | 15.0 | 15.0 | 15.0 | 15.0 | 15.0 | 15.0 | 15.0 |
| SmLs03 | Lower | 15.0 | 15.0 | 15.0 | 15.0 | 15.0 | 15.0 | 15.0 | 15.0 | 15.0 |
| AtmWtAg | Average | 10.2 | 10.9 | 10.2 | 10.9 | 10.2 | 10.3 | 11.2 | 10.2 | 15.0 |
| SmLs04 | Average | 10.1 | 10.3 | 10.1 | 10.3 | 10.4 | 10.7 | 10.6 | 10.1 | 15.0 |
| SmLs05 | Average | 9.9 | 10.3 | 9.9 | 10.3 | 10.2 | 10.5 | 10.6 | 9.9 | 15.0 |
| SmLs06 | Average | 9.9 | 10.3 | 9.9 | 10.3 | 10.2 | 10.5 | 10.6 | 9.9 | 15.0 |
| SmLs07 | Higher | 4.0 | 4.3 | 4.0 | 4.3 | 4.4 | 4.7 | 4.6 | 4.0 | 15.0 |
| SmLs08 | Higher | 3.9 | 4.3 | 3.9 | 4.3 | 4.2 | 4.5 | 4.6 | 3.9 | 15.0 |
| SmLs09 | Higher | 3.9 | 4.3 | 3.9 | 4.3 | 4.2 | 4.4 | 4.6 | 3.9 | 15.0 |
Min*: minimum LRE against the exact statistics of the inputs as parsed into doubles.
On the linear sets the refined QR (equilibrated, pivoted Householder plus double-double
refinement of the augmented system) returns the correctly rounded least-squares solution
of the double inputs everywhere (Min* = 15) except Wampler5’s R^2 = 0.0022, which is formed
as a difference (13.8). Against the certified values the remaining gap is input rounding:
Wampler2’s y values and Pontius’s x values are decimal fractions that binary cannot hold.
Filip reaches 14.0 (McCullough’s “acceptable” bar is 7; normal equations score 0, and
plain Householder QR about 7-8). The simple-regression routine reaches 13.9 on Norris and
14.7 on NoInt1/NoInt2 through the same solver (tests/nist_linear.rs::nist_simple_path).
On the ANOVA sets every statistic is correctly rounded for the double inputs (Min* = 15: sums of squares on data shifted by the first observation, two-pass means, compensated sums). Against the certified values, SmLs04-09 are limited by their inputs: 1000000.4 and 1000000000000.4 are not representable in binary. On the parsed doubles the exact SS_between of SmLs05 is 16.080000001825 (certified 16.08, so at most LRE 9.94) and of SmLs07 1.6801563 (certified 1.68, at most 4.03); no double-precision program can do better on these inputs. AtmWtAg (7 constant digits) reaches 10.2 for the same reason. The tests hold the lower sets to 13, the average sets to 10 (9.9 for the input-capped SmLs05/06) and every set to 13 against the exact statistics of its double inputs.
NIST StRD nonlinear regression (LRE, both starting points)
| Dataset | Difficulty | Start | Status | Iterations | Parameters | SDs | Residual SS | Residual SD | Min |
|---|---|---|---|---|---|---|---|---|---|
| Misra1a | Lower | 1 | Converged | 16 | 11.0 | 10.8 | 10.5 | 10.6 | 10.5 |
| Misra1a | Lower | 2 | Converged | 4 | 11.0 | 10.8 | 10.5 | 10.6 | 10.5 |
| Chwirut2 | Lower | 1 | Converged | 9 | 10.9 | 11.0 | 11.0 | 10.9 | 10.9 |
| Chwirut2 | Lower | 2 | Converged | 6 | 10.9 | 11.0 | 11.0 | 10.9 | 10.9 |
| Chwirut1 | Lower | 1 | Converged | 10 | 10.7 | 10.9 | 11.0 | 10.9 | 10.7 |
| Chwirut1 | Lower | 2 | Converged | 6 | 10.7 | 10.9 | 11.0 | 10.9 | 10.7 |
| Lanczos3 | Lower | 1 | Ambiguous | 79 | 9.7 | 9.7 | 10.6 | 11.0 | 9.7 |
| Lanczos3 | Lower | 2 | Ambiguous | 10 | 9.3 | 9.3 | 10.6 | 11.0 | 9.3 |
| Gauss1 | Lower | 1 | Converged | 5 | 10.6 | 10.6 | 11.0 | 11.0 | 10.6 |
| Gauss1 | Lower | 2 | Converged | 6 | 10.6 | 10.6 | 11.0 | 11.0 | 10.6 |
| Gauss2 | Lower | 1 | Converged | 6 | 10.3 | 10.4 | 10.6 | 10.7 | 10.3 |
| Gauss2 | Lower | 2 | Converged | 6 | 10.3 | 10.4 | 10.6 | 10.7 | 10.3 |
| DanWood | Lower | 1 | Converged | 6 | 11.0 | 11.0 | 11.0 | 11.0 | 11.0 |
| DanWood | Lower | 2 | Converged | 6 | 11.0 | 11.0 | 11.0 | 11.0 | 11.0 |
| Misra1b | Lower | 1 | Converged | 20 | 11.0 | 10.8 | 11.0 | 11.0 | 10.8 |
| Misra1b | Lower | 2 | Converged | 5 | 11.0 | 10.8 | 11.0 | 11.0 | 10.8 |
| Kirby2 | Average | 1 | Converged | 8 | 10.5 | 10.7 | 11.0 | 10.8 | 10.5 |
| Kirby2 | Average | 2 | Converged | 7 | 10.5 | 10.7 | 11.0 | 10.8 | 10.5 |
| Hahn1 | Average | 1 | Ambiguous | 11 | 10.8 | 10.5 | 10.6 | 11.0 | 10.5 |
| Hahn1 | Average | 2 | Ambiguous | 12 | 10.8 | 10.5 | 10.6 | 11.0 | 10.5 |
| Nelson | Average | 1 | Converged | 57 | 10.9 | 10.6 | 10.9 | 11.0 | 10.6 |
| Nelson | Average | 2 | Converged | 12 | 10.9 | 10.6 | 10.9 | 11.0 | 10.6 |
| MGH17 | Average | 1 | Ambiguous | 454 | 10.8 | 10.6 | 11.0 | 11.0 | 10.6 |
| MGH17 | Average | 2 | Ambiguous | 16 | 10.8 | 10.6 | 11.0 | 11.0 | 10.6 |
| Lanczos1 | Average | 1 | Ambiguous | 73 | 10.6 | 3.2 | 2.9 | 3.2 | 2.9 |
| Lanczos1 | Average | 2 | Ambiguous | 8 | 10.6 | 3.2 | 2.9 | 3.2 | 2.9 |
| Lanczos2 | Average | 1 | Ambiguous | 76 | 10.4 | 10.3 | 10.1 | 10.8 | 10.1 |
| Lanczos2 | Average | 2 | Ambiguous | 9 | 10.4 | 10.2 | 10.0 | 10.6 | 10.0 |
| Gauss3 | Average | 1 | Converged | 7 | 10.5 | 10.5 | 11.0 | 10.8 | 10.5 |
| Gauss3 | Average | 2 | Converged | 7 | 10.5 | 10.5 | 11.0 | 10.8 | 10.5 |
| Misra1c | Average | 1 | Converged | 7 | 10.8 | 10.6 | 11.0 | 11.0 | 10.6 |
| Misra1c | Average | 2 | Converged | 4 | 10.8 | 10.6 | 11.0 | 11.0 | 10.6 |
| Misra1d | Average | 1 | Converged | 7 | 11.0 | 11.0 | 11.0 | 11.0 | 11.0 |
| Misra1d | Average | 2 | Converged | 6 | 11.0 | 11.0 | 11.0 | 11.0 | 11.0 |
| Roszman1 | Average | 1 | Converged | 6 | 10.9 | 11.0 | 11.0 | 11.0 | 10.9 |
| Roszman1 | Average | 2 | Converged | 5 | 10.9 | 11.0 | 11.0 | 11.0 | 10.9 |
| ENSO | Average | 1 | Converged | 30 | 10.7 | 10.8 | 11.0 | 11.0 | 10.7 |
| ENSO | Average | 2 | Converged | 28 | 10.7 | 10.8 | 11.0 | 11.0 | 10.7 |
| MGH09 | Higher | 1 | Converged | 395 | 10.5 | 10.4 | 11.0 | 11.0 | 10.4 |
| MGH09 | Higher | 2 | Converged | 26 | 10.5 | 10.4 | 11.0 | 11.0 | 10.4 |
| Thurber | Higher | 1 | Ambiguous | 40 | 10.4 | 10.7 | 11.0 | 10.6 | 10.4 |
| Thurber | Higher | 2 | Ambiguous | 29 | 10.4 | 10.7 | 11.0 | 10.6 | 10.4 |
| BoxBOD | Higher | 1 | Converged | 23 | 11.0 | 10.4 | 10.4 | 11.0 | 10.4 |
| BoxBOD | Higher | 2 | Converged | 11 | 11.0 | 10.4 | 10.4 | 11.0 | 10.4 |
| Rat42 | Higher | 1 | Converged | 10 | 11.0 | 10.6 | 11.0 | 10.4 | 10.4 |
| Rat42 | Higher | 2 | Converged | 6 | 11.0 | 10.6 | 11.0 | 10.4 | 10.4 |
| MGH10 | Higher | 1 | Ambiguous | 253 | 10.9 | 11.0 | 11.0 | 11.0 | 10.9 |
| MGH10 | Higher | 2 | Ambiguous | 116 | 10.9 | 11.0 | 11.0 | 11.0 | 10.9 |
| Eckerle4 | Higher | 1 | Converged | 17 | 10.5 | 10.5 | 10.7 | 11.0 | 10.5 |
| Eckerle4 | Higher | 2 | Converged | 8 | 10.5 | 10.5 | 10.7 | 11.0 | 10.5 |
| Rat43 | Higher | 1 | Converged | 22 | 11.0 | 10.7 | 11.0 | 11.0 | 10.7 |
| Rat43 | Higher | 2 | Converged | 9 | 11.0 | 10.7 | 11.0 | 11.0 | 10.7 |
| Bennett5 | Higher | 1 | Ambiguous | 745 | 11.0 | 10.4 | 11.0 | 10.6 | 10.4 |
| Bennett5 | Higher | 2 | Ambiguous | 196 | 11.0 | 10.4 | 11.0 | 10.6 | 10.4 |
LRE capped at 11 (NIST certifies 11 digits). Parameters and SDs: minimum over the parameters. Status “Ambiguous” is Prism’s flag for a parameter with dependency > 0.9999 (the fit converged; the parameters are nearly redundant). Starts that did not converge: 0. Worst parameter LRE over all 54 fits: 9.33.
Every one of the 27 sets converges from both NIST starting points with plain
Levenberg–Marquardt (MINPACK lmder port: scaled trust region, pivoted QR, Moré’s
lambda update) followed by Newton polishing. No restarts and no geodesic acceleration are
needed; geodesic acceleration (available as FitOptions::geodesic) was tried and is not
the default, because from MGH09’s first start it converges to a different stationary
point. The asked bars were lower >= 7, average >= 6, higher >= 4; the worst parameter LRE
is 9.3 (Lanczos3, whose data carry five significant digits), and 52 of the 54 fits have
every parameter at 10.3 or better. MGH09, MGH10, Bennett5, Eckerle4, Rat43 and BoxBOD
reach 10.4 to 11 from both starts (MGH09 start 1 takes 395 iterations, Bennett5 start 1
745).
Why the polish matters: LM stops on a relative sum-of-squares test, and near the optimum SS(b + d) - SS(b) is of order d^2, so a test on SS can only resolve the parameters to about sqrt(eps). Without the polish, ENSO, MGH09 and Thurber stop at LRE 6.8-7.4. The Newton step solves (J’J + sum r_i H_i) d = -J’r with J’r accumulated in compensated arithmetic and the second derivatives H_i from central differences of the exact Jacobian; a step is accepted while the equilibrated gradient norm falls. That resolves the parameters to about eps x condition, which is what the table shows.
Lanczos1 is the exception, and the limit is its data, not the algorithm. Its
certified residual SS is 1.4307867721e-25: the y values are the model rounded to 13
digits, so the residuals are about 7.7e-14, roughly 100 units in the last place of y.
Parsing the y values into doubles moves each by up to 2.2e-16, which changes the minimum
SS by 6.5e-4 relative. Evaluated in 60-digit decimal arithmetic at Dioptra’s parameters,
the SS is 1.43078687e-25 with the decimal data (certified: 1.43078677e-25) but 1.42986e-25
with the data as parsed into doubles: an LRE of 3.2 is the best any double-precision
program can reach on these inputs. Dioptra’s 2.9 adds the rounding of the model
evaluation in double. The parameters themselves reach 10.6, and the parameter SDs and
residual SD inherit the SS’s 3.2 because they scale with sqrt(SS). tests/nist_nonlinear.rs
holds every other set to 9 on all four quantities from both starts, and Lanczos1’s SS, SDs
and residual SD to 2.5.
Notes on the fixtures
-
Studentized range, far upper tail. The worst relative errors in that row (8e-6) are in the SciPy-generated reference, not in Dioptra. For k = 2 the studentized range has a closed form, P(Q > q) = P(|T_df| > q/sqrt 2). At q = 8, df = 1000 that is 2.0119785627859e-8 (40-digit mpmath and our t distribution agree); the fixture holds 2.0119947042e-8 (SciPy
studentized_range.sf, off by 8.0e-6 relative), and Dioptra gives 2.01197856277e-8 (1e-11). The case passes on the fixture’s absolute tolerance (1e-11).tests/distributions.rs::quadrature_identitiespins k = 2 to the t distribution at 1e-10 relative, and the ignored teststudentized_range_vs_nested_quadraturechecks k = 2..20, df = 1..1000 and inf against nested adaptive quadrature (worst: 1e-13 absolute on the cdf, 1e-11 relative on the upper tail). -
Dunnett. The fixture’s own double integral is accurate to about 2e-9 relative (p_greater = 1.3658528033e-5 for the unequal-n case); nested adaptive quadrature gives 1.36585280593e-5, which Dioptra matches to 3e-14. Dunnett with one treatment equals the t test exactly and is tested that way.
-
ROUT robust centre. The fixture’s centre comes from SciPy’s
minimize_scalar; its merit gradient there is about 1e-8, ours about 1e-13, and the two centres differ by up to 3e-10 relative (tolerance 1e-9). Flagged indices agree everywhere. -
Noncentral t. All six tables pass at 1e-10, including x = -3, df = 5, nc = 8, where the reference is 1.899970783e-20 and SciPy returns 8.0e-20.
-
Shapiro–Wilk. W and p agree with SciPy to 1.3e-13 even though the fixture tolerance is 1e-7 / 1e-6 (the AS R94 polynomials are single precision; we use the full-precision normal scores, as R does, and so does SciPy).
-
NIST NumAcc3/NumAcc4. The SD LREs of 9.5 and 8.3 against the certified 0.1 are the limit set by the inputs: 1000000.1 and 10000000.1 are not representable in binary, and the exact SD of the parsed doubles is 0.10000000003492 and 0.10000000055879. Dioptra reproduces those exact values to 15 digits (starred columns).
-
Spearman, perfect correlation (correlation row). For
n5the ranks agree exactly, so rho = 1 and the t-approximation p is 0. SciPy reports 1.40e-24 because its rho rounds to 0.9999999999999999; Dioptra’s rounds to a slightly different neighbour of 1 and gives a p of the same order. The case passes on the absolute tolerance (1e-10); it is the only reason the correlation row shows a relative error above 1. Every other correlation comparison, including the exact Spearman and Kendall p-values, agrees to 2e-13 relative or better. -
Tukey–Kramer, Dunnett, p-value adjustment. The 2.0e-9 relative error is the fixture’s own Dunnett double integral (see the Dunnett note above); Tukey p-values agree to 8e-12 (the worst is a tail p of 5.3e-5), Tukey critical values and CIs to 1e-13, and all seven p-adjustment methods to 1e-13. The loose library cross-checks stored in the same fixture (statsmodels’
qsturnginterpolation, SciPy’s randomized-QMCdunnettp and CI, tolerance 2e-3) are reported on their own “sanity” row so that they do not mask the exact comparisons; SciPy’s QMC Dunnett p for the unequal-n case is 3.8e-5 against the exact 2.73e-5. -
Two-way ANOVA trap. The fixture’s
type3_treatment_coding_WRONG_for_reference(type III with dummy coding) is checked to differ from Dioptra’s sum-to-zero type III on the unbalanced case; matching it would be the classic error. -
Fisher r x c. The network algorithm is checked against plain enumeration of every table on 60 pseudo-random tables (p to 1e-10, table counts exactly,
contingency::tests::network_matches_enumeration) and matches all five fixture tables, including the transposed 3 x 2 and the 2 x 2 that equalsfisher_exact. The conditional MLE odds ratio and its exact CI agree with SciPy to 1e-13. -
Exact discrete p-values. Runs-test p (Swed–Eisenhart), Kendall and Spearman exact p and Fisher p are compared at 1e-12 relative or tighter where the fixture tolerance is looser, as the tolerance policy asks for rational quantities.
-
Nonlinear regression. The fixture optima come from SciPy
curve_fitandleast_squares, polished with central-difference Jacobians; Dioptra uses exact derivatives (hand-written for the built-in library, forward automatic differentiation for parsed equations). Parameters, SEs, CIs, SS, AICc, R^2 and the covariance agree to 4e-8 relative or better, except two covariance entries of the Gaussian case (Amplitude-Mean and Mean-SD, 2e-10 and 1e-11: zero but for rounding, the X grid being symmetric about the mean), which differ by 3e-4 relative and pass on the absolute tolerance; that is SciPy’s finite-difference Jacobian. Everyfit/case is run twice: through the parsed fixture equation from the fixture’s start, and through the built-in Prism equation (4PL, one phase decay, Michaelis–Menten, specific binding with Hill slope, Gaussian) from the Prism-style initial-value rules; both reach the same optimum. The robust cases (fixed-scale Cauchy and soft-L1) agree to 6e-8. -
Cox regression. lifelines stops its Newton iterations early: its score at its own estimate is -1.6e-6, -2.1e-7 and -4.6e-6 for the three cases, and for
small_censor_at_endits coefficient is 2.8e-6 relative from the maximum (the fixture asks 1e-6). Dioptra’s coefficient equals the maximum found by Newton iterations in 50-digit decimal arithmetic (-0.84574684893160798) to 1e-15, and its log-likelihood is higher than lifelines’ (-17.930671038121 against -17.930671038126). The test therefore checks the coefficient, SE and log-likelihood against the 50-digit maxima at 1e-11, checks that Dioptra’s likelihood is at least lifelines’, and compares the lifelines values at 1e-5; that last comparison is the 4.7e-6 in the survival row. Every KM, Greenwood, confidence-interval, log-rank, Gehan–Breslow–Wilcoxon, Tarone–Ware and hazard-ratio value agrees to 1e-11. -
ROC. The relative error of 1.0 in the ROC row is a Wilson lower limit at x = 0: the fixture holds 1.39e-17 (rounding in NumPy) where the exact value, and Dioptra’s, is 0; it passes on the absolute tolerance. Everything else agrees to 1e-14.
-
Hand-tested (no fixture). ICC: the six Shrout & Fleiss (1979) coefficients of their worked example with R
psych::ICC’s F tests and McGraw–Wong CIs, to the printed digits. Cohen’s kappa and its SEs: a hand-computed 2 x 2 table. Bootstrap: the BCa interval of a normal mean matches the t interval, BCa shifts a skewed variance interval the right way, and results are reproducible from the seed. Permutation test: exact enumeration against a hand count, Monte Carlo against the t test. xoshiro256**: the reference outputs from state (1, 2, 3, 4). DeLong’s paired comparison: identities (a monotone transform gives zero difference and zero SE; swapping the tests flips z). Profile-likelihood CIs equal the t CIs on a straight line to 1e-7 and are right-skewed for a rate constant; a global 4PL fit with shared Top and Bottom recovers noiseless parameters to 1e-9; ROUT on a curve finds the two planted outliers and none in clean data.
Performance (release build, one core)
From cargo test -p dioptra-stats --release --test benches -- --ignored --nocapture:
| Operation | Time |
|---|---|
| betainc(2.5, 7.5, x) body | 0.169 us |
| gammainc_p(4.5, x) body | 0.095 us |
| betainc(50, 50, x) body | 0.198 us |
| betainc_inv(2, 3, p) | 0.845 us |
| norm_quantile(p) | 0.008 us |
| Shapiro-Wilk n=5000 | 95.727 us |
| Mann-Whitney exact 50x50 | 1125.738 us |
| Mann-Whitney exact 100x100 | 14340.008 us |
| StudentizedRange(k=5, df=20).cdf_sf | 1139.164 us |
| StudentizedRange(k=10, df=60).cdf_sf | 1469.011 us |
| StudentizedRange::new(5, 20) (node setup) | 16.721 us |
| StudentizedRange(k=5, df=20).ppf(0.95) | 17279.177 us |
| Dunnett(k=3, unequal).p_value | 7146.145 us |
| Dunnett(k=3, unequal).quantile(0.95) | 76594.293 us |
| NoncentralT(10, 1.5).cdf (AS 243) | 0.331 us |
| NoncentralT(5, 8).cdf(-3) (tail quadrature) | 27.356 us |
| Fisher r x c 3x5, N = 150 | 13.6 ms |
| Fisher r x c 4x5, N = 200 (near independence) | 353 ms |
| Fisher r x c 3x10, N = 200 | 31.7 ms |
| Fisher r x c 5x6, N = 200 (near independence) | 561 ms |
| Fisher r x c 5x6, N = 200, strongly associated | exceeds the work/path limits; returns NoConvergence after 1.0 s, 154 MB peak |
| Tukey-Kramer, k = 10 (45 pairs, critical q) | 75 ms |
| 4PL fit, 1,000 points, from the initial-value rules | 1.37 ms |
| 4PL fit, 1,000 points, with profile CIs and a 1,000-point band table | 42.6 ms |
| NIST Thurber (7 parameters, 37 points) from start 1 / start 2 | 0.83 ms / 0.58 ms |
| Kaplan-Meier, 10,000 subjects | 0.13 ms |
| Log-rank, 2 x 10,000 subjects | 0.23 ms |
| Cox PH (Efron), 20,000 subjects, 1 covariate | 2.7 ms |
The r x c limits are the FISHER_RXC_WORK_LIMIT (5e7 path-child expansions) and
FISHER_RXC_PATH_LIMIT (4e6 stored paths per stage). Tables near independence finish well
inside them up to 5 x 6 with N = 200; strongly associated tables of that size do not (R’s
FEXACT also needs an enlarged workspace or simulation there), and the chi-square test is
the fallback. Prism offers Fisher’s test for 2 x 2 tables only. The Tukey time is almost all
the studentized-range quantile for the simultaneous CI.
Performance
Measured with cargo test -p dioptra-core --release -- --ignored --nocapture bench_
(release profile: thin LTO, opt-level 3) on a 16-core desktop. Times are wall clock for
one Session::dispatch_json call, so they include JSON parsing of the command, the work,
and the undo snapshot.
| Operation | Size | Time | Target |
|---|---|---|---|
| Paste (parse, detect, store) | 1,000 columns × 1,000 rows (10.8 MB of text) | 0.135 s | < 1.5 s |
| Paste (parse, detect, store) | 10 columns × 100,000 rows (10.8 MB of text) | 0.130 s | < 1.5 s |
| Read a 50-row viewport right after the paste | 30 columns × 50 rows | 0.5–1.3 ms | — |
Column formula = A * 2 + 1 (recompute on first read) | 1,000,000 rows | 44 ms | < 150 ms |
Column formula = log10(A) * 100 / (A + 1) | 1,000,000 rows | 105 ms | — |
Column formula = (A - mean(A)) / sd(A) | 1,000,000 rows | 51 ms | — |
Column formula = if(A > 500, A, NA) | 1,000,000 rows | 79 ms | — |
Column formula = lag(A, 1) - A | 1,000,000 rows | 44 ms | — |
How it stays fast:
- Paste parses straight into column vectors of
f64; only text cells allocate. - A formula is compiled once against the table: summaries (
mean(A)) and series (lag(A, 1)) are computed whole, then a small expression tree runs once per row. - Recompute is pull-based and per column: an edit marks downstream formula columns dirty, and they are recomputed when something reads them.
GetCellsmaterialises only the requested viewport.- Undo stores the cells a command touches, or the whole previous table when a paste reshapes it (for a paste into an empty table that copy is nearly free).
Graphs (dioptra-plot)
Measured with cargo test -p dioptra-plot --release --test bench -- --ignored --nocapture
on the same desktop; medians of 9–30 runs after a warm-up. “SVG” is the whole path from
spec and data to the SVG string (family, layout with text measurement, scene, writer).
| Operation | Size | Time | Target |
|---|---|---|---|
| Scatter → SVG | 10,000 points | 1.2–2.3 ms (381 kB) | < 30 ms |
| Scatter → SVG | 100,000 points (decimated to the 0.75 pt grid) | 6.3–6.9 ms (744 kB) | < 300 ms |
| Column graph with points and brackets → SVG | 3 groups × 12 | 0.04 ms | — |
| Volcano → SVG | 1,500 genes, labels placed | 0.24 ms | — |
| Column graph, 85 × 70 mm → PNG | 300 dpi (1004 × 827 px) | 3.8 ms | < 400 ms |
| Column graph, 85 × 70 mm → PDF | fonts embedded as subsets | 1.4 ms (16 kB) | < 300 ms |
| Scatter, 10,000 points → PNG | 300 dpi | 61 ms | — |
| Scatter, 10,000 points → PDF | 126 ms (608 kB) | — |
How it stays fast:
- Text is measured from advance-width tables built once from the bundled fonts; no shaping happens during layout.
- Markers of one series share one
<g>with the paint, so each point is one short<circle>or<path>; numbers are written with two decimals by integer arithmetic. - Above 20,000 points a series keeps one point per 0.75 pt cell (a pixel at 96 dpi) of the plot area, which is invisible at any print size and bounds the SVG; the SVG carries a comment saying how many points were drawn.
- PNG and PDF parse the engine’s own SVG once with usvg against a font database holding only the four bundled Inter files.
Measured by dioptra bench
dioptra bench measures the whole engine through the same JSON commands the app sends,
so these are the numbers a user sees, not those of an isolated function. Run it yourself
with a release build:
cargo build -p dioptra-cli --release
./target/release/dioptra bench # or --markdown for this table
Measured on 2026-10-11 on an AMD Ryzen 7 7800X3D (8 cores, 16 threads), Linux, release build; each time is the median of 3 to 9 runs and includes parsing the command and, for mutations, the undo snapshot.
| Operation | Size | Time (median) |
|---|---|---|
| Paste (parse, detect, store) | 1,000,000 cells, 10 × 100,000 (10.8 MB of text) | 76.2 ms |
| Export table to CSV | 1,000,000 cells | 91.7 ms |
| Export table to XLSX | 1,000,000 cells | 611 ms |
| Save the project (.dioptra) | 1,000,000 cells | 235 ms |
| Column formula = A * 2 + 1 (set, recompute on read) | 1,000,000 rows | 19.6 ms |
| Edit one cell, formula column recomputes | 1,000,000 rows | 17.4 ms |
| Scatter → SVG for the editor (with hit-test targets) | 10,000 points (381 kB) | 2.02 ms |
| Scatter → SVG file | 10,000 points (381 kB) | 1.81 ms |
| Scatter → PDF (fonts embedded) | 10,000 points (549 kB) | 146 ms |
| Scatter → PNG at 300 dpi | 10,000 points (73 kB) | 57.5 ms |
What the rows do:
- Paste: a fresh column table receives 10.8 MB of tab-separated text (10 columns by 100,000 rows); the engine detects separator and decimal mark, parses, and stores every cell.
- Export / save: the same 1,000,000 cells written as CSV, as an Excel workbook, and as
a
.dioptraproject (zip with the data, the analyses and the history). - Column formula: setting
= A * 2 + 1on a second column of 1,000,000 rows and reading a viewport, which recomputes the column; then editing one cell ofA, which marks the formula column dirty, and reading again. - Scatter: 10,000 points rendered for the graph editor (SVG plus the hit-test boxes of every clickable element), then exported as SVG, as PDF with the fonts embedded, and as PNG at 300 dpi.
Architecture and internals
This chapter is for people who want to read or change the code. It explains how the parts
fit, why they are split the way they are, and how to add the two things people most often
want to add: an analysis and a graph family. CONTRIBUTING.md in the repository has the
setup steps and the rules for numerical code.
The shape of the system
apps/dioptra (React UI)
│ Command (JSON) ▲ Response (JSON)
▼ │
┌─ Tauri: invoke("dispatch") ──┐ or ┌─ browser: wasm Engine.dispatch ─┐
└──────────────┬───────────────┘ └───────────────┬─────────────────┘
▼ ▼
dioptra-core ── Session: one dispatcher, the document model
│ │
▼ ▼
dioptra-stats dioptra-plot
(numerics) (scene, SVG, PDF, PNG)
Everything that computes is Rust. The UI is TypeScript and React and never does arithmetic on data. It sends commands and draws responses. This split exists for two reasons. First, one engine means one set of answers: the desktop app, the browser build used in tests and the command line give identical numbers because they run identical code. Second, Rust compiles to native code for the desktop and to WebAssembly for the browser without changes.
The crates
dioptra-stats is the numerical library: special functions (incomplete beta and
gamma, their inverses), distributions (t, F, chi-square, noncentral t, studentised range,
Dunnett), descriptive statistics, t tests, rank tests, ANOVA, post-hoc comparisons,
contingency tables, correlation, linear regression with a refined QR solver,
Levenberg–Marquardt nonlinear regression with the equation library, normality tests,
outliers, survival, ROC, agreement, PCA, bootstrap and permutation. It knows nothing about
tables or files: numbers in, numbers out. Every public function names its reference in
docs/METHODS.md.
dioptra-core is the document model: tables of the eight kinds, columns with
provenance, the formula language (lexer, Pratt parser, evaluator), derived tables, the
dependency graph and pull-based recomputation, analyses as specifications with results
sheets, the command history with inverses for undo, import and export, the .dioptra file,
and the API. compute/ wires each analysis kind to dioptra-stats and writes its results
sheet, including the assumption checks (compute/checks.rs).
dioptra-plot turns a graph specification and extracted data into a scene: a
device-independent list of shapes and text measured in points. Text is measured with the
bundled Inter fonts’ own metrics, so layout never depends on the system. The scene is
written as SVG by the crate’s own writer, as PNG through resvg and as PDF through svg2pdf.
dioptra-wasm is a thin wasm-bindgen facade: an Engine with one method,
dispatch(json) -> json.
dioptra-cli runs the same engine from the shell: dioptra analyze data.csv --kind column --test one_way_anova, and info, run, import, render, export, bench and
equations on a project file, for scripted pipelines and continuous integration.
apps/dioptra is the Tauri 2 desktop app with the React UI. src-tauri/src/lib.rs
exposes one Tauri command, dispatch. src/engine/ holds the bridge with two backends,
Tauri and wasm, behind one interface, and a queue that keeps commands in order.
The Command/Response bridge
The whole API is two enums in crates/dioptra-core/src/api.rs. Command is every request
the UI can make: create a table, paste text at a position, fetch a viewport of cells, set a
column formula, list the analyses that fit a table, create an analysis, get its results
sheet, render a graph to SVG, patch a graph specification, undo, save. Response is every
answer: Ok with what changed, Error with a message, Cells, PasteReport,
Results, Svg with hit-test targets, Bytes for files. Both are serde-tagged JSON.
{"type": "paste", "table": "t1", "row0": 0, "col0": 0,
"text": "1,5\t2,5\n3,0\t4,5", "options": {}}
Session::dispatch_json parses the command, runs it, and serialises the response. Commands
that mutate the project record an inverse for undo and a line in the history.
Why JSON across the bridge instead of typed bindings? Because the same string goes through Tauri’s IPC, through wasm-bindgen, and into a test, and it can be logged and replayed. The cost is a parse per call; at the sizes involved (a viewport of a few hundred cells, a results sheet) it is a fraction of a millisecond, measured in Performance.
Large data never cross the bridge whole. The grid asks for the cells in view
(GetCells with a range), so a million-cell table costs the UI only what is on screen.
The WebAssembly build
scripts/build-wasm.sh compiles dioptra-wasm for wasm32-unknown-unknown and runs
wasm-bindgen into apps/dioptra/src/engine/wasm/pkg. The UI then runs in a plain browser
against the same engine. This build is how the end-to-end tests run: Playwright drives
Chrome through paste, analysis, graph, export, save and reopen, with no desktop window
involved. A hosted web version would use the same build; it is deferred.
Adding an analysis
Take a hypothetical sign test as the example.
- The numerics go in
dioptra-stats: a function from values to a result struct (the statistic, the p value, a confidence interval). Name the reference in its doc comment and indocs/METHODS.md. - The oracle. Add cases to
scripts/oracle/gen_fixtures.pythat compute the same quantities with SciPy (scipy.stats.binomtestfor a sign test), regenerate the fixture JSON, and write a test that reads it and compares at the stated tolerance. Include the awkward cases: ties, zeros, n = 1, all values equal. - The specification. Add a
SignTestSpecwithspec_struct!incrates/dioptra-core/src/analysis.rs, a variant ofAnalysisSpec, an entry inKINDS(the display name), a default indefault_for, and the table kinds it accepts inaccepts. The JSON form is{"kind": "sign_test", ...}. - The computation. Write
fn sign_test(ctx: &AnalysisContext) -> Result<ResultsSheet, String>incrates/dioptra-core/src/compute/and register it inregister_all. Useextractto get clean values with notes about what was left out,checksfor the assumption checks, and the helpers for the methods line, summary sentences in the project’s p-value style, and typed sections. - The UI. Add a one-line description to
apps/dioptra/src/analysis/descriptions.tsand the options toschema.ts; the options panel is generated from the schema. - The documentation. An entry in
docs/ANALYSES.md: when to use it, assumptions, options, outputs, reference, worked example.
Adding a graph family
- Write a module in
crates/dioptra-plot/src/families/withpub fn plot(ctx: &Ctx) -> Plot: read the extracted data and the spec’s options, and return axis plans and layers of marks in data units. The shared layout code measures labels, places axes and legends, and maps everything to points. - Add the name to
FAMILIESand a branch inbuildinfamilies/mod.rs. - Give every layer an id and a target pointer so the graph editor can select it (the table of targets is in the graph reference).
- Add a snapshot test: a golden SVG per theme in
crates/dioptra-plot/tests/snapshots/.UPDATE_SNAPSHOTS=1 cargo test -p dioptra-plotwrites it; look at it before committing. - Add the family to the UI’s list in
apps/dioptra/src/graph/families.tsand document it indocs/GRAPHS.md.
The test strategy
Each layer is tested against the strongest reference available for it.
- Numerics against NIST StRD certified values and oracle fixtures from SciPy, statsmodels, lifelines and scikit-learn, plus identities and textbook examples (see Correctness). Exact quantities are compared at 10⁻¹² relative.
- The document model with unit tests per table kind, golden tests for the formula parser, recompute tests, undo/redo tests over every command, and a round-trip test: save, open, compare, field by field.
- The plot engine with golden SVGs per family and theme, PDF structure checks (header, cross-reference table, embedded fonts), PNG size checks (mm × dpi), and timing benchmarks.
- The UI with TypeScript’s type checker, vitest for pure logic (keyboard navigation, the command registry, option schemas), and Playwright end-to-end tests on the wasm build.
- The desktop build launches under a virtual display in CI to prove it starts and renders.
./verify runs all of it and is the release gate: formatting, clippy with warnings as
errors, every cargo test, the wasm build, type check and unit tests, the frontend and
desktop builds, and the end-to-end suite.
The complaint map
Before any code was written, the project collected what working scientists dislike about
GraphPad Prism and about the spreadsheet step most people take before it. The sources were
GraphPad’s own documentation and knowledge base (which documents many limits candidly),
published papers on spreadsheet errors and data presentation, and review sites. Each
complaint got a number, P1 to P29, and each design decision in Dioptra traces back to one
or more of them. The research, with every source, is research/02-complaints.md in the
repository.
This is not a list of Prism’s failures. Prism is well liked: around 4.6 to 4.7 out of 5 on the review sites. People complain about the edges of a tool they mostly value. Ten things they value (K1 to K10) are listed at the end, and Dioptra keeps all of them.
The story in five parts
Cost and platform (P1–P4). Prism is a subscription, priced per user per year, and some of the features that make statistics honest (effect sizes for ANOVA, in-table formulas) moved to higher tiers in Prism 11. There is no Linux version, and the Windows and Mac versions differ in what they can import and export. Dioptra is MIT-licensed, has no account and no licence server, builds for Linux, Windows and macOS from one codebase, and renders the same bytes on all three.
The spreadsheet step (P5–P13). Most data reach Prism through Excel, and Excel changes data on the way: gene symbols become dates (in 2016 about a fifth of papers with Excel gene lists had such errors), pasted values are the rounded display rather than the stored number, negative numbers formatted in parentheses arrive positive. Prism’s own contingency and survival tables drop decimal points on paste. Reshaping data to fit a table format is manual, and formulas, the main reason Excel stays in the loop, arrived in Prism late and only in one table kind. Dioptra’s answer is to make the spreadsheet step unnecessary: paste raw text, keep text as text, read spreadsheet files directly at full precision, reshape with stored transforms, and compute with formulas in every table.
Statistics (P14–P19). Prism documents assumptions in help pages but does not check them on the results; asterisks are the default summary; methods are under-reported in the literature; estimation graphics came late; nested data are easy to analyse as if every cell were independent. Dioptra puts the checks on the results sheet, leads with the estimate and its interval, writes a methods sentence, draws estimation plots and SuperPlots, and analyses nested tables on the experimental unit.
Records and files (P20–P24). Prism files are hard to inspect, change format between versions, and do not diff. Dioptra’s file is documented JSON in a zip, versioned, diffable after unzip, and carries the full command history.
Figures (P25–P29). Getting a Prism figure into a journal often means Illustrator: releasing clipping masks, fixing fonts. Colour-blind-safe palettes came late; bar charts remain the default; large tables are slow. Dioptra exports SVG with real text and PDF with embedded fonts, sizes figures in millimetres, ships colour-blind-safe palettes by default, shows points on every column graph, and handles a million cells interactively.
The full map
| # | Complaint | Dioptra’s answer |
|---|---|---|
| P1 | Price and subscription licensing | MIT licence, free binaries for every platform, no account, no expiry |
| P2 | Effect sizes and formulas behind higher tiers | Effect sizes with confidence intervals in every results sheet; formulas in every table |
| P3 | No Linux | AppImage, deb and rpm on every release; developed on Linux |
| P4 | Windows and Mac versions differ | One Rust engine and one UI; the same SVG, PDF and PNG bytes on every system (bundled fonts) |
| P5 | Excel round trips corrupt data (gene names to dates) | No Excel step: paste raw text; text stays text, never coerced; import reads stored cell values |
| P6 | Paste copies displayed, rounded values | Full precision stored; display decimals are a column property; xlsx import reads stored values |
| P7 | Contingency and survival tables rewrite “1.23” as “123” | Tables never rewrite input; the analysis validates and says what it expected |
| P8 | Direct Excel import fragile and Windows-only | Built-in reader for xlsx, xls and ods, identical on every system, no Office needed |
| P9 | Choosing the table format is hard | Each kind explained in one sentence with a sketch; the paste dialog suggests a kind; Multiple variables is the safe default |
| P10 | Replicates laid out differently per table | Transforms convert between layouts (stack, pivot, average replicates) with the recipe stored |
| P11 | Long (tidy) data must be reshaped by hand | Multiple variables tables are first-class; pivot long to wide is one derived table |
| P12 | No formulas where expected | Column and cell formulas in a formula bar, dependency tracked, recomputed on edit |
| P13 | Missing values break designs | Missing is a value; each analysis states how it handled missing data; mixed models are planned |
| P14 | P-hacking by dialog | Every analysis is an object in the project and every run is in the history, including deleted ones |
| P15 | Assumptions documented, not checked | Assumption checks run with the analysis, shown first, with the alternative named |
| P16 | Asterisks and “significant” as the default | APA-style exact p values by default; the summary leads with the estimate and CI; asterisks optional |
| P17 | Methods under-reported | Every results sheet has a methods sentence (test, variant, sidedness, correction, version) |
| P18 | Estimation came late and only for t tests | Estimation plots with bootstrap CIs of the difference |
| P19 | Hierarchical data analysed as independent | Nested tables analyse unit means and say so; the default graph is a SuperPlot |
| P20 | No scripted record of the analysis | Formulas, specifications and the history in JSON; recompute reproduces every number; the dioptra command line re-runs a project file and renders its graphs |
| P21 | Opaque file format | .dioptra is documented JSON in a zip (File format) |
| P22 | Version lock-in | Versioned format; newer readers open every older file; CSV and XLSX export always |
| P23 | No version control | Stable-ordered, pretty-printed JSON that diffs after unzip; line-oriented history |
| P24 | Collaboration is file passing | An open file makes git-based collaboration possible; real-time sync is not planned |
| P25 | Export needs Illustrator workarounds | SVG with real text and no clipping masks; PDF with embedded subset fonts |
| P26 | Figures finished in other software | Sizes in mm and points, journal presets, multi-panel layouts with panel letters |
| P27 | Limited colour and theme control | Designed themes, colour-blind-safe palettes by default, per-series colours |
| P28 | Modern distribution plots late; bars the default | Violin, raincloud, estimation, before–after and SuperPlot; column graphs show points by default |
| P29 | Slow or unstable with large tables | Columnar storage, viewport fetches, a 10⁶-cell paste in about 0.13 s (Performance) |
What is kept from Prism
| # | What people like | How Dioptra keeps it |
|---|---|---|
| K1 | Guided analysis by table kind | The Analyze list shows what fits the table and says why the rest does not |
| K2 | Results linked to data | Edit a value and every dependent analysis and graph recomputes |
| K3 | Graphs that follow the data | Graphs are specifications over tables and analyses, not pictures |
| K4 | Publication-quality defaults | Designed themes, typographic sizes in points, bundled fonts |
| K5 | The analysis checklist | Before you report this on every results sheet |
| K6 | Documentation that teaches | This manual explains each default and each test |
| K7 | Nonlinear regression done well | The equation library, constraints, shared parameters, model comparison, ROUT, profile CIs |
| K8 | Raw and summary data both accepted | Mean/SD/N and other summary formats in XY and grouped tables |
| K9 | Excluded values stay visible | Struck through, kept, counted |
| K10 | Works offline, no code | A desktop program; nothing leaves your machine |
Keyboard shortcuts
Every command is in the command palette: press Ctrl-K (⌘-K on a Mac), type a few letters of what you want, press Enter. The palette shows each command’s shortcut, so it is also the way to learn them. F1 shows this list inside the app.
On macOS read ⌘ for Ctrl and ⌥ (Option) for Alt.
File
| Command | Shortcut |
|---|---|
| New project | Ctrl-Alt-N |
| New table | Ctrl-Alt-T |
| Open | Ctrl-O |
| Save | Ctrl-S |
| Save as | Ctrl-Shift-S |
| Import file (CSV, TSV, XLSX, XLS, ODS, PZFX) | Ctrl-I |
| Paste as new table | Ctrl-Shift-V |
| Export table (CSV, TSV, XLSX) | Ctrl-Shift-E |
Edit
| Command | Shortcut |
|---|---|
| Undo | Ctrl-Z |
| Redo | Ctrl-Shift-Z or Ctrl-Y |
| Cut, copy, paste | Ctrl-X, Ctrl-C, Ctrl-V |
| Edit cell | F2 or Enter |
| Fill down | Ctrl-D |
| Exclude or include the selected values | Ctrl-E |
| Clear | Delete or Backspace |
| Select all | Ctrl-A |
| Recompute everything | F9 |
Insert
| Command | Shortcut |
|---|---|
| Rows above | Ctrl-Alt-I |
| Delete rows | Ctrl-Alt-minus |
| Add a data set | Ctrl-Alt-D |
Analyze and graph
| Command | Shortcut |
|---|---|
| Analyze | Ctrl-Shift-A |
| New graph | Ctrl-Shift-G |
| Export graph | Ctrl-Alt-E |
View
| Command | Shortcut |
|---|---|
| Command palette | Ctrl-K |
| Toggle dark theme | Ctrl-Shift-L |
| Show or hide the project rail | Ctrl-B |
| Show or hide the inspector | Ctrl-\ |
| Focus the grid | Ctrl-Alt-G |
| Close tab | Ctrl-Alt-W |
| Keyboard shortcuts | F1 |
Moving in the grid
| Key | Moves |
|---|---|
| Arrow keys | One cell |
| Shift-arrow | Extends the selection |
| Tab, Shift-Tab | Right, left; wraps to the next or previous row |
| Enter, Shift-Enter | Down, up (after typing, commits and moves) |
| Home, End | Start, end of the row |
| Ctrl-Home, Ctrl-End | First cell, last cell of the table |
| Page Up, Page Down | One screen |
| Escape | Cancels an edit |
While typing in a cell, the arrow keys commit the value and move, as in a spreadsheet. In the formula bar, Enter commits and Escape cancels.
FAQ
Does it run on Linux?
Yes, as a first-class platform: it is developed on Linux. Each release has an AppImage (any
distribution), a .deb (Debian, Ubuntu, Mint) and an .rpm (Fedora, RHEL, openSUSE). See
Getting started.
What does it cost?
Nothing. Dioptra is free and open source under the MIT licence. There is no account, no licence server, no trial period and no tier. Every analysis, effect size and export format is in the one version there is.
Can I use it at work, in a company, or in a clinical setting?
The MIT licence allows any use, commercial included, without asking. It also comes with no warranty, as all open-source licences do. For regulated work, validate it as you would any software: the correctness record and the test suite are there to make that easier.
Can I import my Excel files?
Yes. File → Import reads .xlsx, .xls and .ods directly, with no Office
installation, the same way on every system. It reads the stored values at full precision,
not the rounded values the sheet displays. Dates become ISO dates. You can also paste
straight from Excel; then the clipboard carries what is displayed, so show enough decimals
first.
Can I open my Prism files?
Prism’s XML format, .pzfx, and the Prism 10 format, .prism, can be imported: the data
tables come across. Prism’s analyses and graphs are not converted; re-run the analyses in
Dioptra. The old binary .pzf format is not read; if you have only that, open it in Prism
and save it as .pzfx or .prism. Dioptra does not
write Prism formats; it exports CSV, TSV and XLSX for anyone who needs the data elsewhere.
Where are my data? Does anything leave my computer?
Your data live in the .dioptra files you save, wherever you save them, and nowhere else.
Dioptra makes no network connections: no telemetry, no analytics, no update check, no
cloud. It works with the network cable unplugged.
How do I know an old analysis will give the same result next year?
The project file stores the data, every formula as text, every analysis as a
specification, every graph as a specification, and the version of Dioptra that wrote it.
Opening it recomputes everything from the data. The file format is versioned and newer
versions read older files. And because the file is documented JSON, the data and the
recipe can be read without Dioptra at all, by unzip and a text editor. See
File format.
Why is Welch’s t test the default instead of Student’s?
Student’s t test assumes the two groups have the same variance. When they do, Welch’s test gives nearly the same answer. When they do not, especially with unequal group sizes, Student’s test can give a p value that is far too small or far too large. Welch costs almost nothing when the assumption holds and protects you when it does not. Student’s test is one option away. The longer argument is in Analyses.
Can I run a one-tailed test?
Yes; every test with a direction has the option. It is not the default because a one-tailed test is valid only when the direction was decided before the data were collected and an effect in the other direction would be treated as no effect. When you choose it, the methods sentence says so, so a reader knows.
Why are there no asterisks on my graphs?
Because the default p-value style is APA: brackets show p = .012, not *. Asterisks
reduce a continuous measure of evidence to a grade, and readers stop at the grade. If your
journal or lab needs them, set the project’s p-value style to GP (inspector, Project,
P values); every sheet and bracket switches at once, and switching back costs nothing.
How large a data set can it handle?
A table of a million cells pastes in about 0.13 seconds and scrolls without lag, because the grid fetches only the cells on screen. A column formula over a million rows computes in under a tenth of a second. Graphs with more than 20,000 points are thinned for drawing (one point per 0.75 pt cell, invisible at print size) but never for analysis. The measurements are in Performance.
Are the statistics right?
They are checked against the strongest references available: the NIST Statistical Reference Datasets, whose answers are certified to 15 digits, and the established implementations in SciPy, statsmodels, lifelines and scikit-learn. All 58 NIST data sets pass, and 7,841 comparisons in 300 oracle cases agree within their stated tolerances. The whole record, including the places where a reference library turned out to be the less accurate one, is in Correctness. If you find a disagreement, please open a numerical discrepancy issue with the data and the reference value.
What is missing compared with Prism?
Mixed-effects models, three-way ANOVA, two-way ANOVA with repeated measures on both factors, multiple logistic and Cox regression, and a few smaller analyses. They are listed, with the reason each waits, in the roadmap.
How do I cite Dioptra?
The repository has a CITATION.cff file; GitHub’s Cite this repository button turns it
into APA or BibTeX. Cite the version you used (it is in every methods sentence), for
example:
ZahakJ. Dioptra: data, statistics and publication graphs without the spreadsheet (version 0.4.0) [Computer software]. 2026. https://github.com/ZahakJ/dioptra
How can I help?
Report bugs and numerical discrepancies as issues, with a small file that shows the
problem. Contributions are welcome; read CONTRIBUTING.md in the repository first, because
numerical code has rules: a named reference and a test against a known value for every
change.
Roadmap
What comes next, in order, and what waits, with the reason for each. The changelog lists what each release contains.
Next: statistics for real designs
Designs that need more than one-factor repeated measures and unit means.
- Mixed-effects models (REML), so repeated measures with missing values and nested data with several levels are analysed as what they are, with a model-formula layer under the guided dialogs.
- Nested t test and nested one-way ANOVA as mixed models, replacing the analysis of unit means.
- Three-way ANOVA and two-way ANOVA with repeated measures on both factors.
- Multiple linear, logistic and Poisson regression, and Cox proportional hazards regression, on a shared iteratively reweighted least squares and Newton infrastructure.
- Interpolation from standard curves, area under the curve, and row statistics.
Then: extension and scale
- A scripting surface (Python or Lua).
- A plugin system for analyses and graph families.
- Faster rendering for very large scatter plots (more than a million points) with a canvas or WebGL renderer, if real use shows the need.
- A system font picker for figures. Arbitrary fonts can already be embedded in PDF; the missing part is the interface.
Deferred, with reasons
| Item | Why it waits | What would unblock it |
|---|---|---|
| WebGL or canvas rendering for more than 10⁶ points | The engine’s SVG with level-of-detail thinning covers typical data sizes; a second renderer doubles what must be kept in sync | A profiling case from real use |
| Hosted web app | The WebAssembly build exists and runs the end-to-end tests; hosting adds accounts, storage and support | A decision to run a service |
| Mixed-effects models, nested ANOVA inference | Need REML and a model-formula layer; nested tables ship with descriptives, unit-mean analyses and SuperPlots | Statistics for real designs |
| Multiple logistic regression, Cox regression | Need IRLS and Newton infrastructure | Statistics for real designs |
| Three-way ANOVA, two-way repeated measures on both factors | Two-way ANOVA has one repeated factor | Statistics for real designs |
| Scripting (Python or Lua) | The formula language and the file format cover reproducibility; a scripting surface needs a stable API | Extension and scale |
| Collaboration and cloud sync | The file diffs cleanly under git; real-time sync is out of scope | Not planned |
| Plugin system | Premature before the analysis API settles | Extension and scale |
| System font picker in figures | Bundled Inter keeps output identical on every system | The interface work |
| Localised interface | English only | Translators |
| Arrowheads on connecting lines | Prism offers them; needs a marker primitive in the scene and in the PDF path | A request |
| “Leave gap at symbols” for connecting lines | Points are drawn over the lines, which reads the same at journal sizes | A case where it does not |
| Geometric mean as a summary and for summary lines | Needs a log-scale summary through extraction, error bars and the inspector together | A request |
| Different styles per data set on one graph (bars for A, dots for B) | Families draw every data set alike; mixing needs per-data-set family overrides | Real figures that need it |
| Fill patterns on boxes and violins | Boxes and notched boxes are polygons; patterns are clipped to rectangles today | Clipping to polygons |
| Space before the first and after the last category, space for blank cells | group_width covers the gap between categories | A request |
| Labels of points by row title on before–after graphs | Callouts exist for scatter and volcano; categorical families do not use them | A request |
| Error-bar cap styles beyond a width (Prism’s short/long) and error-bar line style | cap_pt covers the width; a dashed error bar is rare | A request |
| Axis origin options beyond “at the edges” and “at zero” | Prism also offers “custom” crossing points | A request |
Suggestions are welcome as GitHub issues. A request that comes with a data set and the answer another program gives is the easiest kind to act on.
Changelog
All notable changes to Dioptra. Versions follow semantic versioning; the file format has its own version (see the manual’s File format chapter).
0.4.0 — 2026-10-14
Prism files both ways, plate readers, and Dioptra Lab.
Free
- Prism import reads
.prismfiles (Prism 10 and later) as well as.pzfx. Nested and multiple-variables tables keep their kind; subcolumn titles are kept; CV, low–high and limits formats are converted, with a note; calendar dates become row titles. Tested on 53 real Prism files from Prism 6 to 11. - Plate-reader import: SoftMax Pro (
.txt), BMG MARS (CSV), Agilent BioTek Gen5 (.txt) and Tecan Magellan (.xlsx), endpoint and kinetic, with an optional plate map of wells to groups and doses. File → Import file… recognises them by content. - Replace table data… loads a new file into an existing table and keeps its formulas, analyses and graphs.
- Verify records: the audit trail and signatures of a regulated project can be checked, viewed and exported without a license.
- Fixed outlines (0.4–0.6 pt) now follow the graph’s line width.
Dioptra Lab
A yearly license for labs, core facilities and regulated groups; see the pricing page. Keys are checked on the computer; nothing is sent anywhere. Help → Dioptra Lab… takes the key.
- Batch analysis: apply a project as a template to many data files, with a summary
table and figures per file; a watch folder for instrument exports;
dioptra batch. - Journal figures: presets for nine journals, a figure check (text size, line width, resolution, colours colour-blind readers cannot tell apart, mismatched panels), TIFF export, layouts at final size with panel letters, a methods paragraph, and a Word or PDF report of the project.
- Prism export: File → Export to Prism… writes data tables as
.pzfx;dioptra convertconverts folders in either direction. - Records for regulated labs: users and roles, a hash-chained audit trail with reasons,
electronic signatures that lock a record, and a validation pack (compliance matrix,
traceability, IQ/OQ protocols) with an automated OQ of 8,212 checks (Records → Run OQ…,
dioptra oq).
File format
dioptra/1.2: layouts gain style; files may carry extensions/<name>. Dioptra 0.3
opens 1.2 files.
Command line
dioptra batch, dioptra convert, dioptra oq, dioptra license.
0.3.0 — 2026-10-12
Connecting lines: Prism’s before–after graphs, made an option of every bar graph.
Graphs
- Connecting lines on column, strip, box and violin graphs (
options.connecting): a line per subject joining the values of each row across the bars, the means or medians joined across the bars, or both. Lines are drawn above bars, boxes and violins and beneath the points; the summary line on top. Style (solid, dashed, dotted), width, opacity and colour: one colour, by data set, per subject from a palette, or by a grouping column with a legend. A missing value leaves a gap (or the line joins across it). On grouped tables the lines run within each group across its data sets (matched subcolumns) or along each data set across the groups. With jitter on, each line ends at its own jittered point. Lines are cut where they cross an axis break. Every subject line is a target in the graph editor. - Paired t, ratio paired t, Wilcoxon, repeated-measures ANOVA and Friedman now graph as bars with connecting lines; the results sheet’s button says Graph with connecting lines. The New graph dialog offers Bars with connecting lines and Before–after (points) as presets for column and grouped tables.
- Horizontal column, strip, box and violin graphs (
options.orientation);axes.ystays the value axis, as in Prism. - Bar options from a comparison with Prism’s Format Graph: error-bar direction (both, above, below) and cap width; bar width and slot width (the gaps between bars and between groups); bar fill (tint, solid, outline); fill patterns per data set (diagonals, crosshatch, horizontal, vertical, dots), drawn as geometry so SVG, PDF and PNG agree; strip graphs without points (mean ± error only) or without the mean line.
- Open markers (
circle_openand friends) now draw open on column graphs. - Any line now breaks where it crosses an axis break, and horizontal bars are cut at breaks of the value axis.
- New inspector sections: Connecting lines and Bars and points (orientation, summary, jitter, fill, widths, error-bar direction and caps); a pattern control per data set.
File format
- Format 1.1. Graph specs gain the fields above; defaults are not written, so graphs
without them are saved as before. The
before_afterfamily is retired: opening a 1.0 file turns such graphs into the strip spec that draws the same picture (unjittered points, subject lines, the mean line when it was on).docs/FILE-FORMAT.mddocuments the migration.
Research and docs
research/06-connecting-lines.md: what Prism’s column, grouped and categorical graphs offer for connecting lines, with sources, and how Dioptra maps each option.- Manual: Connecting lines and paired graphs, with a worked example.
0.2.1 — 2026-10-12
Fixed
- macOS: the app opened as “Dioptra is damaged and can’t be opened”. The bundle is now
ad-hoc signed as a whole (
bundle.macOS.signingIdentity: "-"), and the release job fails unlesscodesign --verify --deep --strictpasses on the.appinside the.dmg. The app is still not notarized: the first launch needs right-click → Open (macOS 15 and later: Privacy & Security → Open Anyway) orxattr -cr /Applications/Dioptra.app. Seedocs/macos-gatekeeper.md.
0.2.0 — 2026-10-12
The design release: the same engine, a redesigned instrument.
Interface
- A design system (
DESIGN.md§12): warm paper neutrals in light, deep graphite in dark, one verdigris accent, Inter for the interface and JetBrains Mono with tabular figures for every number, an 8-pt spacing scale, Lucide icons, designed focus rings and thin themed scrollbars. Every text/background pair meets WCAG AA in both themes (docs/DESIGN-AUDIT.md,scripts/contrast.mjs). - No native form control anywhere: own Select, Segmented, Toggle, Checkbox, NumberField with unit suffixes, ColorSwatch, Menu, Dialog and Tooltip.
- The in-window menu bar is gone. A slim top bar carries the mark, the project title (click to rename), the command field and the theme control; the menu tree lives in a compact Menu popover and the command palette. macOS gets a native system menu.
- Dark mode follows the operating system (and the Tauri window theme) by default, with a light/dark/system control in the top bar, View and the palette; every surface is designed for dark rather than inverted.
- The grid is a bounded sheet: rows end where the data ends plus a margin, with “+ rows” and “+ Data set” affordances; data-set headers carry a kind glyph and the colour swatch the graph will use; a designed selection with corner handles; comfortable and compact density; a first-class formula bar; human-readable provenance times.
- Results sheets in a reading column with the type system, hairline tables, mono numbers, assumption cards with designed marks and the alternative on a second line.
- The graph editor is a desk with millimetre rulers, a paper choice (white, theme, transparent), a designed zoom control, hover halos and a selection frame on targets, and collapsible inspector sections that remember their state.
- A designed first launch: paste data, open the bundled sample project, or read the manual.
- A new icon set from the dioptra mark; window size, position and panel widths are remembered.
- The paste dialog’s segmented choices sit at their own width instead of stretching across the column.
- A designed README banner (also the landing page’s Open Graph image), and a landing page whose captures follow its light or dark theme.
Engine
SetProjectTitlecommand (undoable); the risk-difference Wald interval is clamped to [-1, 1].
0.1.0 — 2026-10-11
The first public release.
Data tables
- Eight table kinds: XY, Column, Grouped, Contingency, Survival, Parts of whole, Multiple variables and Nested, each explained with a sketch when a table is created.
- Subcolumn formats: replicate values, mean/SD/N, mean/SEM/N, mean/CI/N, mean/SD, mean/SEM, median/IQR, lower/upper and single values.
- A virtualised grid that holds a million cells, with the full spreadsheet keyboard set, fill down, undo and redo across every action, and a status line with n, mean, SD and sum.
- Text cells kept as text, never coerced; excluded values struck through and kept.
- Cell provenance: typed, pasted (with time), imported (file, sheet, column), formula or derived, plus a per-cell edit log.
Getting data in and out
- Paste with automatic separator, decimal-comma and header detection, and a report of what
was decided; ambiguous values such as
1,234are kept as text and reported. - Paste as a new table with a suggested table kind.
- Import of CSV, TSV, text, XLSX, XLS, ODS (built-in reader, stored values at full
precision) and Prism
.pzfxdata tables. - Export of tables to CSV, TSV and XLSX.
Formulas and transforms
- Column formulas and cell formulas with fill-down, a dependency graph, pull-based recomputation and circular-reference detection.
- Arithmetic, logic, row functions, summaries, series functions (
lag,lead,diff,cumsum,rank,zscore,baseline), slices, and unit conversion including mass-to-molar conversion for glucose, cholesterol, triglycerides, creatinine and urea. - Derived tables with stored recipes: transpose, pivot long to wide and wide to long, stack, filter, sort, select rows and columns, remove baseline, fraction of total, normalise, average replicates and prune.
Command line
dioptra info,run,import,analyze,render,export,benchandequationsdrive the same engine from scripts and continuous integration.
Analyses
- Twenty analyses: descriptive statistics; t tests (one-sample, Welch, Student, paired, ratio paired); Mann–Whitney; Wilcoxon signed-rank; one-way ANOVA with Welch and Brown–Forsythe; repeated-measures one-way ANOVA with Greenhouse–Geisser and Huynh–Feldt; two-way ANOVA (types I, II, III; one repeated factor); Kruskal–Wallis; Friedman; chi-square; Fisher’s exact test (2 × 2 and r × c); correlation (Pearson, Spearman, Kendall); simple linear regression with bands and diagnostics; nonlinear regression; normality tests; outlier identification (Grubbs, ROUT); survival (Kaplan–Meier, log-rank, Gehan–Breslow–Wilcoxon, hazard ratios); ROC curves; Bland–Altman; principal component analysis.
- Multiple comparisons: Tukey–Kramer, Dunnett, Šidák, Holm, Bonferroni, Dunn and Benjamini–Hochberg.
- Nonlinear regression with the equation library (4PL, 5PL, exponential, Michaelis–Menten, Hill, binding, Gaussian, polynomials and more), custom equations, weighting, robust fitting, constraints, shared parameters, profile-likelihood intervals, model comparison by extra sum of squares and AICc, and ROUT.
- Results sheets with assumption checks shown first, APA-style summary sentences, a methods sentence, effect sizes with confidence intervals, and a reporting checklist.
- P-value styles APA (default), NEJM, GP and exact, applied project-wide.
- Analyses listed by fit to the table, with the reason for each that does not fit.
Graphs
- Twenty-three families: scatter, line, column, box, violin, strip, raincloud, before–after, estimation, SuperPlot, histogram, density, heatmap, correlation matrix, Kaplan–Meier with at-risk table, ROC, Bland–Altman, volcano, forest, pie, donut, QQ and residual plots; column graphs can be grouped, stacked or percent-stacked.
- An interactive editor: click any element to edit it in the inspector, drag the legend and annotations, edit text in place; significance brackets, reference lines and labels.
- Themes Instrument, Night and Journal; palettes Okabe–Ito (default), Tol bright, muted and light, viridis, cividis, Tol sunset and grey.
- Sizes in millimetres, type and lines in points, journal presets; multi-panel layouts with panel letters.
- Export to SVG (real text), PDF (embedded subset fonts) and PNG at any resolution; identical bytes on every platform.
Files and reproducibility
- The
.dioptraproject file: a zip of documented, pretty-printed JSON with the data, formulas, analysis and graph specifications, and the command history.
Correctness and performance
- All 58 NIST StRD data sets used (univariate, linear regression, ANOVA, nonlinear regression) pass; 7,841 comparisons against SciPy, statsmodels, lifelines and scikit-learn agree within tolerance.
- A million-cell paste in about 0.13 s; a column formula over a million rows in under 0.11 s.
Platforms
- Linux (AppImage, deb, rpm), Windows (msi, setup exe) and macOS (Apple silicon and Intel dmg), built from one codebase.