Tabalyst Report
Open the demo on the synthetic insurance file.
Tabalyst ProductsAvailable
Tabalyst Generate creates synthetic CSV datasets for tests, demos and teaching: believable values, related files and, when you ask for them, defects that are injected on purpose and logged cell by cell. It runs on your machine, in the same package as Tabalyst Report.
tabalyst generate --example computer --rows 20000 -o pcs.csv --anomaly-profile dirty_realistic
Created: pcs.csv
Created: pcs.changes.csv
20,000 rows in 4.6s (4,391 rows/s, 3 processes), 5,672 cells changed: generating rows 4.0s, injecting errors 0.6s
tabalyst generate --example insurance --rows 3000 -d insurance-output/
Created: insurance-output/insurance-main.csv
Created: insurance-output/insurance-ontario.csv
Created: insurance-output/insurance-quebec.csv
Created: insurance-output/insurance-combined.csv
Done in 0.9s
Real output of tabalyst 0.8.0, paths shortened. Times were measured on one machine with four processors.
Real data is hard to share, and a demo needs data that looks real: believable names, addresses, amounts and dates, and the kind of problems that real exports have. A synthetic dataset gives you that without exposing anyone.
Packaged examples for CRM contacts, insurance customers and computer offers, each with the structure and the rules of its domain.
A package fixes its seed and its reference date; --seed and --as-of-date override them. Generation is reproducible with the same package, options and environment.
An anomaly profile writes an altered CSV beside the clean one, a log of every changed cell and a JSON summary of what was requested, applied and skipped, so a profile of the data can be checked against what was injected. Name several profiles to get one variant for each level of errors, from the same rows.
--rows sets the combined total, shared across the branches. Read the insurance customers page.gaming or enterprise, and measurements as plain numbers or with units. Read the computer offers page, which details the 40, 70 and 111 columns.dataset.yaml, a catalog.yaml and the schemas and tables they reference. --validate-only checks it without writing anything. The package format is experimental during beta.Existing files are never overwritten without --force.
An anomaly profile such as dirty_realistic or dirty_stress keeps the clean CSV and adds the altered one, a *.changes.csv log and a summary.
tabalyst generate --example insurance --rows 3000 --anomaly-profile dirty_realistic -d insurance-dirty/For Computer, -o writes one CSV that holds the errors, with its .changes.csv log beside it. With -d, name several profiles to get one variant for each level of errors from a single generation of the rows, so every variant describes the same offers:
tabalyst generate --example computer --rows 30000 -d out/ --anomaly-profile dirty_realistic,dirty_stressComputer rows are generated by several processes from 5,000 rows (--workers sets the number), and the command shows its progress, then ends with a timing report. The same seed gives the same files whatever the number of processes. On a machine with four processors, 1,000,000 offers with a profile of errors took about three minutes.
The synthetic insurance file of the Tabalyst Report demo was made this way: 100,000 customers and 34 columns with the dirty_stress profile, which gives each kind of defect a fixed number of rows. Tabalyst Report then analyzes it as it is, with its real gaps and duplicates.
Open the demo on the synthetic insurance file.
Next step
Install Tabalyst 0.8.0 or later with pip install --upgrade tabalyst (0.7.0 is enough for the examples and packages, 0.7.3 adds the single Computer file with errors, several profiles and the progress), then read the walkthrough or the command reference. To analyze a file, use Tabalyst Report.
Interactive HTML reports and JSON profiles from your files.
Tabalyst Inspect, Tabalyst Scan and Tabalyst Sample.