Dataset report

insurance-customers-100k.csv

  • CSV
  • 100,000 rows
  • 34 columns
  • 25.1 MB
  • utf-8-sig
  • Delimiter ,
Analysis complete · 13.07 s

01 / Overview

Dataset overview

Size, completeness and the main signal to check first.

Rows
100,000No empty rows
Columns
343,400,000 cells analyzed
Missing cells
12.42%422,169 cells
Duplicate rows
8Beyond the first occurrence
Finding types · 34 columns checked

Affected columns

Bar = largest finding share in the column

Errors and Issues

What the checks found, by kind

Column types

Classification: family, then its types

  • Identifier2

    • ID 2
  • Categorical15

    • Enum 14
    • Constant 1
  • Text4

    • Very short string 1
    • Short string 3
  • Temporal6

    • Date 6
  • Numeric4

    • Integer 2
    • Number 2
  • Geographic1

    • Postal code 1
  • Contact2

    • Email 1
    • Phone 1

Technical type: what the values are made of

  • Text 24
  • Date 6
  • Integer 2
  • Number 2

02 / Summary

Column summary(34)

Type, overview and findings per column. Errors: values a rule proves invalid. Issues: ambiguities and deviations to inspect. Missing values are counted apart. Remarks are under Observations.

34 of 34 columns
Column
Type family
Type
Overview
Missing
Errors
Issues
Details
1customer_idIdentifierID99,992 distinct values across at least 9 recurring formats.Unique after row deduplication–Not applicable
CLI-00000696CLI-00006855CLI-00024687
2titleCategoricalEnum2 categories; the most frequent holds 50.0% of present values.–Not applicableMs.Mr.
3first_nameTextVery short stringVery short text, with a median length of 5 characters and 136 distinct values.0.01%7Not applicable
LiamOliviaEmma
4last_nameTextShort stringShort text, with a median length of 6 characters and 220 distinct values.–Not applicable
SmithBrownWilson
5birth_dateTemporalDateDates range from 1940-01-01 to 2008-09-14, in 4 recognized formats (ambiguous dates left out of the range).0.01%6
1945-04-021963-04-151979-05-09
6genderCategoricalEnum2 categories; the most frequent holds 50.0% of present values.–Not applicableFM
7preferred_languageCategoricalConstantOne value on all 100,000 present positions.–Not applicableen
8street_numberNumericIntegerWhole numbers ranging from 1 to 9,999, with a median of 5,006.–58423637403
9street_typeCategoricalEnum15 categories; the most frequent holds 22.2% of present values.–Not applicableStreetAvenueBoulevard
10street_nameTextShort stringShort text, with a median length of 7 characters and 123 distinct values.–Not applicable
MapleCedarPine
11street_directionCategoricalEnum4 categories; the most frequent holds 29.1% of present values.75.07%75,070Not applicableWestNorthSouth
12unit_numberNumericIntegerWhole numbers ranging from 1 to 999, with a median of 499.65.0%65,00336119680
13cityTextShort stringShort text, with a median length of 8 characters and 139 distinct values.–Not applicable
MontréalQuébecLaval
14province_codeCategoricalEnum18 categories; the most frequent holds 75.2% of present values.–Not applicable
QCONAB
15province_nameCategoricalEnum16 categories; the most frequent holds 75.2% of present values.–Not applicable
QuebecOntarioAlberta
16postal_codeGeographicPostal codePostal-code-shaped values in 2 recognized formats.–
H0A 7S5H0A 8T3H0A 3H9
17country_codeCategoricalEnum2 categories; the most frequent holds more than 99.9% of present values.–Not applicable
CAca
18emailContactEmailEmail-shaped values.9.29%9,295
ameliascott@yahoo.exampleanderson.ellie.2@yahoo.examplehenry.taylor.4@hotmail.example
19phoneContactPhonePhone-shaped values in 2 recognized formats.–
+1-200-555-0176+1-200-555-0177+1-200-555-0170
20contract_idIdentifierID99,983 distinct values across at least 3 recurring formats.0.01%9Not applicable
CTR-1000000696CTR-1000006855CTR-1000024687
21contract_typeCategoricalEnum9 categories; the most frequent holds 40.8% of present values.–Not applicable
autohometravel
22start_dateTemporalDateDates range from 2016-09-18 to 2027-08-15, in 5 recognized formats (ambiguous dates left out of the range).–
2026-05-122026-09-102026-09-14
23end_dateTemporalDateDates range from 2016-10-24 to 9999-12-31, in 5 recognized formats (ambiguous dates left out of the range).13.75%13,750
2026-10-172026-10-162026-11-02
24contract_statusCategoricalEnum7 categories; the most frequent holds 64.1% of present values.–Not applicableactiveexpiredsuspended
25annual_premiumNumericNumberNumbers ranging from -10,152 to 10,916.49, with a median of 1,439.98.–
205.881829.04235.13
26payment_frequencyCategoricalEnum6 categories; the most frequent holds 65.1% of present values.–Not applicablemonthlyannualquarterly
27payment_methodCategoricalEnum9 categories; the most frequent holds 63.7% of present values.–Not applicabledirect_debitcredit_cardbank_transfer
28agency_codeCategoricalEnum30 categories; the most frequent holds 27.1% of present values.–Not applicable
AG-PR-001AG-PR-002AG-PR-003
29creation_dateTemporalDateDates range from 2016-08-20 to 2026-09-15, in 5 recognized formats (ambiguous dates left out of the range).–
2026-07-242026-03-272026-07-29
30modification_dateTemporalDateDates range from 2016-08-30 to 2026-09-16, in 5 recognized formats (ambiguous dates left out of the range).–
2026-09-162026-09-142026-09-15
31claim_dateTemporalDateDates range from 2016-10-09 to 2026-09-16, in 3 recognized formats (ambiguous dates left out of the range).85.92%85,915
2026-09-152026-09-142026-09-16
32claim_amountNumericNumberNumbers ranging from -12,868.67 to 446,411.23, with a median of 4,212.52.87.19%87,1870.002090.7010392.53
33claim_statusCategoricalEnum4 categories; the most frequent holds 74.4% of present values.85.92%85,915Not applicableclosedin_progressrejected
34branchCategoricalEnum3 categories; the most frequent holds 72.7% of present values.–Not applicablemainquebecontario

03 / Findings

Observations(42)

What the scan found, by rule, from its facts and never from a guess. A candidate anomaly is worth a look, a remark may be perfectly valid. Counts are exact unless marked.

Confirmed issues
0Defects of the source
Candidate anomalies
20Worth a look
Remarks
22May be valid
Limits
7What the scan could not establish
42 of 42 findings
Status
Finding
Column
Count
Out of
Accuracy
Detail
Candidate anomalyDuplicate rows beyond their first occurrenceduplicate_rowsdataset8100,000recordsexactRecords 1109, 7800, 25601, 47930, 54883…
Candidate anomalyValues written in several ways that normalization compares as equalvariant_groupsfirst_name8–exactMiamia8 groups
Candidate anomalyValues written in several ways that normalization compares as equalvariant_groupslast_name12–exactSmithsmith12 groups
Candidate anomalyDate values matching more than one day-month orderambiguous_datesbirth_date9,84899,994valuesexactDD-MM-YYYYMM-DD-YYYY9,846, +1 more
Candidate anomalyValues written in several ways that normalization compares as equalvariant_groupsstreet_name8–exactCedarcedar8 groups
Candidate anomalyValues written in several ways that normalization compares as equalvariant_groupscity7–exactQuébecQuebec7 groups
Candidate anomalyValues written in several ways that normalization compares as equalvariant_groupsprovince_code5–exactQCqc5 groups
Candidate anomalyValues written in several ways that normalization compares as equalvariant_groupsprovince_name3–exactQuebecquebec3 groups
Candidate anomalyValues written in several ways that normalization compares as equalvariant_groupspostal_code32–exactH0A 3M2H0A 3M232 groups
Candidate anomalyRare lengths among otherwise uniform valuesminority_lengthpostal_code21100,000text valuesexact6 characters
Candidate anomalyValues written in several ways that normalization compares as equalvariant_groupscountry_code1–exactCAca1 group
Candidate anomalyRare lengths among otherwise uniform valuesminority_lengthcontract_id3699,991text valuesexact13 characters15 characters
Candidate anomalyValues written in several ways that normalization compares as equalvariant_groupscontract_type1–exactautoAUTO1 group
Candidate anomalyDate values matching more than one day-month orderambiguous_datesstart_date9,862100,000valuesexactDD-MM-YYYYMM-DD-YYYY9,861, +1 more
Candidate anomalyDate values matching more than one day-month orderambiguous_datesend_date8,45786,250valuesexactDD-MM-YYYYMM-DD-YYYY8,455, +1 more
Candidate anomalyValues outside the type of their columntype_outliersannual_premium1499,997valuesexact–
Candidate anomalyRare lengths among otherwise uniform valuesminority_lengthagency_code16100,000text valuesexact10 characters
Candidate anomalyDate values matching more than one day-month orderambiguous_datescreation_date9,900100,000valuesexactDD-MM-YYYYMM-DD-YYYY9,899, +1 more
Candidate anomalyDate values matching more than one day-month orderambiguous_datesmodification_date9,902100,000valuesexactDD-MM-YYYYMM-DD-YYYY9,902
Candidate anomalyDate values matching more than one day-month orderambiguous_datesclaim_date1,43914,085valuesexactDD-MM-YYYYMM-DD-YYYY1,439
RemarkMissing valuesmissing_valuesfirst_name7100,000valuesexact–
RemarkCells changed by collapsing repeated internal whitespacecollapsed_whitespacefirst_name699,993text valuesexactAbigai··l (whitespace repeated inside the text)Isab··ella (whitespace repeated inside the text)+1 more
RemarkMissing valuesmissing_valueslast_name4100,000valuesexact–
RemarkCells changed by collapsing repeated internal whitespacecollapsed_whitespacelast_name599,996text valuesexactRob··erts (whitespace repeated inside the text)Sm··ith (whitespace repeated inside the text)+1 more
RemarkMissing valuesmissing_valuesbirth_date6100,000valuesexact–
RemarkColumns with one distinct valueconstant_columnpreferred_language100,000100,000valuesexact–
RemarkCells changed by trimming surrounding whitespacetrimmed_cellsstreet_name2100,000text valuesexact…s·tres·longues·rues·sans·fin·chemin·des·tres·longues·rues·sans· (1 trailing whitespace character)
RemarkCells changed by collapsing repeated internal whitespacecollapsed_whitespacestreet_name8100,000text valuesexactNor··thgate (whitespace repeated inside the text)Hea··ther (whitespace repeated inside the text)+1 more
RemarkMissing valuesmissing_valuesstreet_direction75,070100,000valuesexact–
RemarkMissing valuesmissing_valuesunit_number65,003100,000valuesexact–
RemarkMissing valuesmissing_valuesprovince_code3100,000valuesexact–
RemarkCells changed by collapsing repeated internal whitespacecollapsed_whitespacepostal_code54100,000text valuesexactH1J··7P4 (whitespace repeated inside the text)K0G··8R2 (whitespace repeated inside the text)+1 more
RemarkMissing valuesmissing_valuesemail9,295100,000valuesexact–
RemarkCells changed by collapsing repeated internal whitespacecollapsed_whitespacephone42100,000text valuesexact+1··200-555-0171 (whitespace repeated inside the text)+1··201-555-0170 (whitespace repeated inside the text)+1 more
RemarkMissing valuesmissing_valuescontract_id9100,000valuesexact–
RemarkMissing valuesmissing_valuescontract_type2100,000valuesexact–
RemarkCells changed by trimming surrounding whitespacetrimmed_cellscontract_type299,998text valuesexactauto· (1 trailing whitespace character)
RemarkMissing valuesmissing_valuesend_date13,750100,000valuesexact–
RemarkMissing valuesmissing_valuesannual_premium3100,000valuesexact–
RemarkMissing valuesmissing_valuesclaim_date85,915100,000valuesexact–
RemarkMissing valuesmissing_valuesclaim_amount87,187100,000valuesexact–
RemarkMissing valuesmissing_valuesclaim_status85,915100,000valuesexact–

Accuracy: exact counted on every value of the file; at least a scan limit stopped the count; sample counted on a sample of the distinct values. Remarks such as missing values are not quality issues.

04 / Structure

Columns(34)

Inferred type and classification per column. Issues: confirmed issues and candidate anomalies found by the scan; remarks such as missing values are under Observations.

34 of 34 columns
Column
Missing
Distinct
Inferred type
Type
Error
Issues
Examples
1customer_id–99,992textID–0CLI-00000696CLI-00006855CLI-00024687
2title–2textEnum–0Ms.Mr.
3first_name0.01%7136textText / Very short string–1+2 remarksLiamOliviaEmma
4last_name–220textText / Short string–1+2 remarksSmithBrownWilson
5birth_date0.01%642,666dateDate–1+1 remarks1945-04-021963-04-151979-05-09
6gender–2textEnum–0FM
7preferred_language–1textConstant–0+1 remarksen
8street_number–9,999integerInteger–058423637403
9street_type–15textEnum–0StreetAvenueBoulevard
10street_name–123textText / Short string–1+2 remarksMapleCedarPine
11street_direction75.07%75,0704textEnum–0+1 remarksWestNorthSouth
12unit_number65.0%65,003999integerInteger–0+1 remarks36119680
13city–139textText / Short string–1MontréalQuébecLaval
14province_code–18textEnum–1+1 remarksQCONAB
15province_name–16textEnum–1QuebecOntarioAlberta
16postal_code–57,510textPostal code–2+1 remarksH0A 7S5H0A 8T3H0A 3H9
17country_code–2textEnum–1CAca
18email9.29%9,29590,698textEmail–0+1 remarksameliascott@yahoo.exampleanderson.ellie.2@yahoo.examplehenry.taylor.4@hotmail.example
19phone–1,217textPhone–0+1 remarks+1-200-555-0176+1-200-555-0177+1-200-555-0170
20contract_id0.01%999,983textID–1+1 remarksCTR-1000000696CTR-1000006855CTR-1000024687
21contract_type–9textEnum–1+2 remarksautohometravel
22start_date–9,315dateDate–12026-05-122026-09-102026-09-14
23end_date13.75%13,7506,248dateDate–1+1 remarks2026-10-172026-10-162026-11-02
24contract_status–7textEnum–0activeexpiredsuspended
25annual_premium–86,913numberNumber0.01%141+1 remarks205.881829.04235.13
26payment_frequency–6textEnum–0monthlyannualquarterly
27payment_method–9textEnum–0direct_debitcredit_cardbank_transfer
28agency_code–30textEnum–1AG-PR-001AG-PR-002AG-PR-003
29creation_date–9,355dateDate–12026-07-242026-03-272026-07-29
30modification_date–7,759dateDate–12026-09-162026-09-142026-09-15
31claim_date85.92%85,9154,874dateDate–1+1 remarks2026-09-152026-09-142026-09-16
32claim_amount87.19%87,18712,733numberNumber–0+1 remarks0.002090.7010392.53
33claim_status85.92%85,9154textEnum–0+1 remarksclosedin_progressrejected
34branch–3textEnum–0mainquebecontario

Distinct values are exact unless marked: ≥ at least (a scan limit stopped the count), ≈ estimated from a sample with its error.

06 / Cleaning

Transformations(34)

Occurrences changed by each normalization stage, and the spellings it groups. Raw preview values remain unchanged.

34 of 34 columns
Column
Unicode composed
Trimmed
Whitespace collapsed
Case folded
Accents removed
Distinct
Compared distinct
Variant groups
1customer_id–––100.0%100,000–99,99299,9920
2title–––100.0%100,000–220
3first_name––0.01%699.98%99,983–1361288
4last_name––0.01%599.98%99,9820.83%83222020912
5birth_date–––––42,66642,6660
6gender–––100.0%100,000–220
7preferred_language–––––110
8street_number–––––9,9999,9990
9street_type–––100.0%100,000–15150
10street_name––0.01%899.98%99,985–1231158
11street_direction–––24.93%24,930–440
12unit_number–––––9999990
13city–––99.99%99,99428.32%28,3181391327
14province_code–––99.97%99,974–18135
15province_name–––99.99%99,988–16133
16postal_code––0.05%54100.0%100,000–57,51057,51032
17country_code–––99.98%99,978–211
18email–––––90,69890,6980
19phone––0.04%42––1,2171,2170
20contract_id–––99.99%99,991–99,98399,9830
21contract_type–––0.01%9–981
22start_date–––––9,3159,3150
23end_date–––––6,2486,2480
24contract_status–––0.01%5–770
25annual_premium–––––86,91386,9130
26payment_frequency–––––660
27payment_method–––0.01%8–990
28agency_code–––100.0%100,000–30300
29creation_date–––––9,3559,3550
30modification_date–––––7,7597,7590
31claim_date–––––4,8744,8740
32claim_amount–––––12,73312,7330
33claim_status–––––440
34branch–––––330

Variant groups(77)

Raw spellings that every normalization stage compares as equal: an analytical equivalence, not proof of identity. Whitespace is shown as written.

07 / Numeric

Numeric analysis(4)

Range and distribution statistics for accepted numeric values.

4 of 4 columns
Column
Range
Minimum
Maximum
Mean
Median
Distinct
Examples
8street_number9,99819,9994,999.22565,0069,99958423637403
12unit_number9981999500.419249999936119680
25annual_premium21,068.49-10,15210,916.491,704.02631,439.9886,913205.881829.04235.13
32claim_amount459,279.9-12,868.67446,411.238,530.32484,212.5212,7330.002090.7010392.53

08 / Dates

Date analysis(6)

Strict date parsing keeps ambiguous, invalid and non-date values separate. Ambiguous values are never resolved from the other values of the column.

6 of 6 date columns
Column
Status
Breakdown
Valid
Ambiguous
Invalid
Other
Distinct
Formats
5birth_datemixed90.15%90,1469.85%9,848––42,6664 variants
22start_datemixed90.14%90,1389.86%9,862––9,3155 variants
23end_datemixed90.19%77,7939.81%8,457––6,2485 variants
29creation_datemixed90.1%90,1009.9%9,900––9,3555 variants
30modification_datemixed90.1%90,0989.9%9,902––7,7595 variants
31claim_datemixed89.78%12,64610.22%1,439––4,8743 variants

Distribution over time exact

Valid dates counted by the second pass per calendar period, as written: the offset of a datetime is not converted, and ambiguous dates are counted apart and never placed, unless the unambiguous dates of the column prove one order and the second pass reads them with it.

start_date · 90,138 dates by month, 2016-09 to 2027-08 · 9,862 ambiguous not placed
PeriodDates
2016-09259
2016-10533
2016-11540
2016-12591
2017-01510
2017-02495
2017-03522
2017-04537
2017-05539
2017-06546
2017-07550
2017-08609
2017-09545
2017-10525
2017-11524
2017-12547
2018-01582
2018-02529
2018-03560
2018-04568
2018-05535
2018-06561
2018-07589
2018-08540
2018-09569
2018-10559
2018-11547
2018-12556
2019-01557
2019-02480
2019-03550
2019-04552
2019-05559
2019-06543
2019-07564
2019-08585
2019-09601
2019-10562
2019-11552
2019-12596
2020-01564
2020-02513
2020-03605
2020-04556
2020-05592
2020-06613
2020-07585
2020-08557
2020-09582
2020-10602
2020-11566
2020-12582
2021-01569
2021-02519
2021-03580
2021-04589
2021-05608
2021-06585
2021-07603
2021-08634
2021-09591
2021-10584
2021-11589
2021-12608
2022-01622
2022-02523
2022-03557
2022-04599
2022-05604
2022-06596
2022-07684
2022-08641
2022-09621
2022-10594
2022-11605
2022-12623
2023-01639
2023-02580
2023-03622
2023-04668
2023-05669
2023-06641
2023-07664
2023-08601
2023-09835
2023-101,067
2023-111,019
2023-121,159
2024-011,042
2024-021,023
2024-031,069
2024-041,038
2024-051,125
2024-061,027
2024-071,156
2024-081,129
2024-091,072
2024-101,109
2024-111,169
2024-121,145
2025-011,106
2025-02994
2025-031,087
2025-041,116
2025-051,205
2025-061,115
2025-071,178
2025-081,189
2025-091,178
2025-101,209
2025-111,166
2025-121,162
2026-011,257
2026-021,085
2026-031,295
2026-041,240
2026-051,373
2026-061,242
2026-071,423
2026-081,492
2026-09730
2026-101
2026-122
2027-032
2027-042
2027-053
2027-061
2027-071
2027-083
creation_date · 90,100 dates by month, 2016-08 to 2026-09 · 9,900 ambiguous not placed
PeriodDates
2016-0876
2016-09432
2016-10570
2016-11547
2016-12517
2017-01521
2017-02497
2017-03575
2017-04515
2017-05535
2017-06527
2017-07564
2017-08592
2017-09523
2017-10531
2017-11533
2017-12563
2018-01573
2018-02509
2018-03582
2018-04550
2018-05543
2018-06577
2018-07558
2018-08578
2018-09548
2018-10554
2018-11583
2018-12513
2019-01543
2019-02470
2019-03532
2019-04574
2019-05562
2019-06548
2019-07580
2019-08591
2019-09567
2019-10557
2019-11579
2019-12586
2020-01526
2020-02532
2020-03626
2020-04552
2020-05640
2020-06550
2020-07578
2020-08575
2020-09577
2020-10609
2020-11578
2020-12561
2021-01583
2021-02511
2021-03611
2021-04576
2021-05596
2021-06549
2021-07613
2021-08647
2021-09582
2021-10595
2021-11591
2021-12604
2022-01636
2022-02505
2022-03645
2022-04543
2022-05627
2022-06631
2022-07665
2022-08632
2022-09584
2022-10636
2022-11614
2022-12641
2023-01628
2023-02567
2023-03609
2023-04647
2023-05676
2023-06662
2023-07627
2023-08664
2023-091,003
2023-101,049
2023-111,098
2023-121,131
2024-011,071
2024-02953
2024-031,057
2024-041,083
2024-051,120
2024-061,044
2024-071,154
2024-081,126
2024-091,043
2024-101,179
2024-111,135
2024-121,153
2025-011,092
2025-02987
2025-031,098
2025-041,182
2025-051,164
2025-061,116
2025-071,193
2025-081,205
2025-091,180
2025-101,182
2025-111,171
2025-121,235
2026-011,206
2026-021,126
2026-031,244
2026-041,307
2026-051,340
2026-061,292
2026-071,461
2026-081,315
2026-09229
modification_date · 90,098 dates by month, 2016-08 to 2026-09 · 9,902 ambiguous not placed
PeriodDates
2016-081
2016-092
2016-104
2016-1110
2016-1213
2017-0115
2017-0216
2017-0319
2017-0426
2017-0523
2017-0634
2017-0732
2017-0853
2017-0940
2017-1060
2017-1137
2017-1260
2018-0159
2018-0268
2018-0370
2018-0484
2018-05100
2018-0684
2018-0780
2018-08101
2018-09125
2018-10126
2018-11119
2018-12144
2019-01121
2019-02135
2019-03144
2019-04148
2019-05137
2019-06166
2019-07181
2019-08185
2019-09152
2019-10191
2019-11179
2019-12223
2020-01207
2020-02192
2020-03231
2020-04188
2020-05220
2020-06219
2020-07231
2020-08230
2020-09282
2020-10288
2020-11262
2020-12268
2021-01308
2021-02269
2021-03284
2021-04287
2021-05316
2021-06312
2021-07331
2021-08363
2021-09338
2021-10396
2021-11358
2021-12394
2022-01415
2022-02375
2022-03460
2022-04383
2022-05424
2022-06488
2022-07489
2022-08478
2022-09470
2022-10501
2022-11529
2022-12535
2023-01560
2023-02488
2023-03557
2023-04608
2023-05620
2023-06576
2023-07643
2023-08682
2023-09654
2023-10699
2023-11736
2023-12768
2024-01819
2024-02819
2024-03844
2024-04921
2024-05996
2024-06953
2024-071,133
2024-081,157
2024-091,099
2024-101,156
2024-111,203
2024-121,345
2025-011,369
2025-021,320
2025-031,547
2025-041,541
2025-051,666
2025-061,631
2025-071,857
2025-081,863
2025-091,942
2025-102,095
2025-112,144
2025-122,321
2026-012,570
2026-022,416
2026-032,973
2026-043,064
2026-053,493
2026-063,833
2026-074,706
2026-086,072
2026-093,751
claim_date · 12,646 dates by month, 2016-10 to 2026-09 · 1,439 ambiguous not placed
PeriodDates
2016-102
2016-114
2016-121
2017-015
2017-023
2017-034
2017-047
2017-056
2017-065
2017-077
2017-0810
2017-0918
2017-1014
2017-1110
2017-1221
2018-0116
2018-0221
2018-0319
2018-0425
2018-0514
2018-0622
2018-0719
2018-0827
2018-0916
2018-1024
2018-1119
2018-1227
2019-0126
2019-0227
2019-0327
2019-0422
2019-0532
2019-0630
2019-0742
2019-0829
2019-0938
2019-1037
2019-1126
2019-1227
2020-0146
2020-0233
2020-0350
2020-0437
2020-0554
2020-0647
2020-0756
2020-0842
2020-0945
2020-1052
2020-1142
2020-1264
2021-0151
2021-0254
2021-0358
2021-0460
2021-0562
2021-0664
2021-0777
2021-0875
2021-0960
2021-1063
2021-1177
2021-1266
2022-0183
2022-0256
2022-0383
2022-0475
2022-0582
2022-0673
2022-0775
2022-0892
2022-0986
2022-1092
2022-11113
2022-1279
2023-01104
2023-0283
2023-0390
2023-04109
2023-05108
2023-0697
2023-0797
2023-08103
2023-09124
2023-10126
2023-11129
2023-12150
2024-01159
2024-02151
2024-03147
2024-04173
2024-05187
2024-06185
2024-07168
2024-08179
2024-09194
2024-10178
2024-11207
2024-12173
2025-01220
2025-02171
2025-03221
2025-04197
2025-05213
2025-06232
2025-07243
2025-08239
2025-09278
2025-10239
2025-11267
2025-12285
2026-01294
2026-02264
2026-03292
2026-04334
2026-05390
2026-06393
2026-07461
2026-08551
2026-09388

09 / Text

String analysis(24)

Length classes, fixed widths and representative values.

24 of 24 string columns
Column
Class
Fixed
Min
Max
Mean
Median
Lengths
Distinct
Examples
1customer_idshort–91311.8212.099,992CLI-00000696ONT0000402
2titlevery short3––––2Ms.
3first_nameshort–3105.575.0136OliviaLiamJames
4last_nameshort–2105.816.0220SmithWilsonCampbell
6gendervery short1––––2F
7preferred_languagevery short2––––1en
9street_typeshort–395.896.015StreetDriveRoad
10street_namevery long–22997.267.0123
11street_directionvery short–454.55.04NorthWest
13citymedium–4259.078.0139MontréalQuébecSherbrooke
14province_codevery short2––––18QC
15province_namemedium–5256.776.016QuebecOntarioBritish Columbia
16postal_codeshort–677.07.057,510H0A 7S5
17country_codevery short2––––2CA
18emailmedium–163825.1825.090,698aanderson.2@hotmail.exampleaanderson.11@gmail.exampleameliascott@yahoo.example
19phoneshort15––––1,217+1-200-555-0176
20contract_idshort–131514.014.099,983CTR-1000000696CTR-000007224CTR--1000009580
21contract_typeshort–484.344.09autotraveltravel_X
24contract_statusshort–6116.546.07activeexpiredsuspended
26payment_frequencyshort–6117.127.06monthlyannualquarterly
27payment_methodshort–51411.6512.09direct_debitcredit_cardbank_transfer
28agency_codeshort–9109.09.030AG-PR-001AG--PR-002
33claim_statusshort–4116.646.04closedin_progressrejected
34branchshort–474.644.03mainquebecontario

10 / Detectors

Detectors and formats(29)

What each detector recognized per column, with the formats it found. Primary: the interpretation the classification starts from.

29 of 29 detections
Column
Detector
Matched
Ambiguous
Invalid
Formats
2titleenumeration primary100.0%100,00000–
5birth_datedate90.15%90,1469,8480YYYY-MM-DDDD-MM-YYYYMM-DD-YYYYYYYY/MM/DD
6genderenumeration primary100.0%100,00000–
7preferred_languageenumeration primary100.0%100,00000–
8street_numbernumber primary100.0%100,000000
9street_typeenumeration primary100.0%100,00000–
11street_directionenumeration primary100.0%24,93000–
12unit_numbernumber primary100.0%34,997000
14province_codeenumeration primary100.0%99,99700–
15province_nameenumeration primary100.0%100,00000–
16postal_codepostal_code primary99.98%99,98400A9A 9A9A9A9A9
17country_codeenumeration primary100.0%100,00000–
18emailemail primary99.99%90,693012–
19phonephone primary100.0%100,00000+1-999-999-9999+1 999-999-9999
21contract_typeenumeration primary100.0%99,99800–
22start_datedate90.14%90,1389,8620YYYY-MM-DDDD-MM-YYYYMM-DD-YYYYYYYY/MM/DDDD/MM/YYYY
23end_datedate90.19%77,7938,4570YYYY-MM-DDDD-MM-YYYYMM-DD-YYYYYYYY/MM/DDMM/DD/YYYY
24contract_statusenumeration primary100.0%100,00000–
25annual_premiumnumber primary99.99%99,983000.0
25annual_premiumcurrency–000.0 CAD
26payment_frequencyenumeration primary100.0%100,00000–
27payment_methodenumeration primary100.0%100,00000–
28agency_codeenumeration primary100.0%100,00000–
29creation_datedate90.1%90,1009,9000YYYY-MM-DDDD-MM-YYYYMM-DD-YYYYYYYY/MM/DDMM/DD/YYYY
30modification_datedate90.1%90,0989,9020YYYY-MM-DDDD-MM-YYYYMM-DD-YYYYYYYY/MM/DDDD/MM/YYYY
31claim_datedate89.78%12,6461,4390YYYY-MM-DDDD-MM-YYYYMM-DD-YYYY
32claim_amountnumber primary100.0%12,813000.0
33claim_statusenumeration primary100.0%14,08500–
34branchenumeration primary100.0%100,00000–

11 / Shapes

Patterns(15)

Structural patterns inferred from the distinct values the scan sampled, grouped by family (the values of one structure); the column page shows each family with its nodes and tables. 9 a digit, A a capital, a a lowercase letter, @ any letter, X a capital or a digit, x a lowercase letter or a digit, # a letter or a digit, other characters as written. Shares count distinct values, never occurrences in the file, and are not validated against the whole file.

Columns with patterns
15of 34 columns
Without patterns
19Numbers, dates, masked or too few distinct values
Grammar
v2Version of the pattern rules
114 of 114 patterns
Column
Family
Pattern
Role
Distinct values
Share
Basis
Validated on every value
Length
Examples
1customer_idUDf2'ONT000'9999principal10042.19%100sampleof sampled distinct values9.09%9,09010ONT0007694ONT0002225ONT0007991
1customer_idUP-Df1'CLI-000'99999secondary8636.29%86sampleof sampled distinct values72.7%72,70112CLI-00064850CLI-00008317CLI-00033746
1customer_idUP-Df1'CLI-00'99999secondary145.91%14sampleof sampled distinct values0.01%1411not kept
1customer_idUP-DP-Df3'QC-000'9-9999secondary145.91%14sampleof sampled distinct values18.17%18,17212QC-0001-8146QC-0000-4550QC-0001-0983
1customer_idUP-Df1'CLI--000'99999secondary125.06%12sampleof sampled distinct values0.01%1213not kept
1customer_idUP-DP-Df3'QC--000'9-9999secondary52.11%5sampleof sampled distinct values0.01%513not kept
1customer_idUP-DP-Df3'QC-000'9-999secondary41.69%4sampleof sampled distinct values–11not kept
1customer_idUDf2AAA999999secondary10.42%1sampleof sampled distinct values–9not kept
1customer_idUP-DP-Df3AA-999-9999secondary10.42%1sampleof sampled distinct values–11not kept
2titleULP.f1'M'a.principal2100.0%2exactof every distinct value100.0%100,0003Ms.Mr.
3first_nameULf1Aaaaaprincipal2921.32%29sampleof sampled distinct values22.69%22,6915JacobEllieMatex
3first_nameULf1Aaaaaasecondary2719.85%27sampleof sampled distinct values24.97%24,9726SamuelAmeliaEvelyn
3first_nameULf1Aaaasecondary2216.18%22sampleof sampled distinct values23.77%23,7734LucaNovaLuvi
3first_nameULf1Aa{7}secondary118.09%11sampleof sampled distinct values8.57%8,5738TheodoreBenjaminScarlett
3first_nameULf1Aaaaaaasecondary118.09%11sampleof sampled distinct values8.1%8,1047MatthewJacksonAbigail
3first_nameULf1Aa{8}secondary42.94%4sampleof sampled distinct values6.33%6,3349CharlotteSebastian
3first_nameULf1Aaasecondary42.94%4sampleof sampled distinct values5.52%5,5183ZoeAvaLeo
3first_nameLf2aaaasecondary42.94%4sampleof sampled distinct values–4ezramilaaleb
3first_nameULf1AAaaaaasecondary32.21%3sampleof sampled distinct values–7JJulianOOliverAAmelia
3first_nameLf2aaaaasecondary21.47%2sampleof sampled distinct values–5not kept
3first_nameLf2a{7}secondary21.47%2sampleof sampled distinct values–7william
3first_nameULP Lf3Aaaa' 'asecondary21.47%2sampleof sampled distinct values–7Davi d
3first_nameLULf4aAaaaasecondary21.47%2sampleof sampled distinct values–6oSphia
3first_nameULf1Aa{9}secondary10.74%1sampleof sampled distinct values–10not kept
3first_nameLf2aaasecondary10.74%1sampleof sampled distinct values–3mia
3first_nameLf2aaaaaasecondary10.74%1sampleof sampled distinct values–6harper
3first_nameULP Lf3Aa aaaasecondary10.74%1sampleof sampled distinct values–7Ha rper
3first_nameULP Lf3Aa aaaaaasecondary10.74%1sampleof sampled distinct values–9Vi ctoria
3first_nameULP Lf3Aaa aaasecondary10.74%1sampleof sampled distinct values–7not kept
3first_nameULP Lf3Aaaa' 'aaaasecondary10.74%1sampleof sampled distinct values–10Isab ella
3first_nameULP Lf3Aaaa' 'aaaaasecondary10.74%1sampleof sampled distinct values–11Alex ander
3first_nameULP Lf3Aaaaa' 'asecondary10.74%1sampleof sampled distinct values–8not kept
3first_nameULP Lf3Aaaaaa' 'asecondary10.74%1sampleof sampled distinct values–9Abigai l
3first_nameLULf4aAasecondary10.74%1sampleof sampled distinct values–3iMa
3first_nameLULf4aAaaasecondary10.74%1sampleof sampled distinct values–5mEily
3first_nameLULf4aAa{7}secondary10.74%1sampleof sampled distinct values–9not kept
4last_nameULf1Aaaaaaprincipal4319.46%43sampleof sampled distinct values25.5%25,4946BaileyRenaudWlaker
4last_nameULf1Aaaaasecondary3214.48%32sampleof sampled distinct values32.12%32,1205AhmdeWlsonSingh
4last_nameULf1Aa{7}secondary2812.67%28sampleof sampled distinct values15.75%15,7518LachanceGxuthierBergeron
4last_nameULf1Aaaasecondary198.6%19sampleof sampled distinct values10.29%10,2934KingKhanSato
4last_nameULf1Aaaaaaasecondary198.6%19sampleof sampled distinct values8.05%8,0457MichaudOuelletThompon
4last_nameULf1Aaasecondary83.62%8sampleof sampled distinct values2.84%2,8443AliLiiRoy
4last_nameULf1Aa{8}secondary62.71%6sampleof sampled distinct values1.59%1,5889RodriguezAndersonnFernandes
4last_nameLULf2aAaaasecondary52.26%5sampleof sampled distinct values0.01%65rBown
4last_nameLULf2aAaaaaaasecondary52.26%5sampleof sampled distinct values0.01%88aCmpbell
4last_nameLf3aaaaaasecondary52.26%5sampleof sampled distinct values0.01%56turnertaylorwalker
4last_nameULf1Aasecondary41.81%4sampleof sampled distinct values1.39%1,3922WuLe
4last_nameLULf2aAaaaasecondary41.81%4sampleof sampled distinct values–6aTylorgNuyen
4last_nameLf3aaaaasecondary41.81%4sampleof sampled distinct values–5clarkartel
4last_nameULf1Aa{9}secondary31.36%3sampleof sampled distinct values0.67%67210RichardqonDesjardins
4last_nameLULf2aAaasecondary31.36%3sampleof sampled distinct values–4not kept
4last_nameLf3a{7}secondary20.9%2sampleof sampled distinct values–7hompson
4last_nameLf3a{8}secondary20.9%2sampleof sampled distinct values–8not kept
4last_nameULP Lf4Aa' 'aaasecondary20.9%2sampleof sampled distinct values–7Sm ithEv ans
4last_nameXf5@@@@secondary20.9%2sampleof sampled distinct values0.3%3044not kept
4last_nameXf5@{8}secondary20.9%2sampleof sampled distinct values0.31%3108not kept
4last_nameUP'ULf7A''''Aaaaasecondary20.9%2sampleof sampled distinct values0.4%3987D'SouzaO'Brien
4last_nameLUf8aAsecondary20.9%2sampleof sampled distinct values–2oH
4last_nameULf1AAaaaasecondary10.45%1sampleof sampled distinct values–6not kept
4last_nameULf1AAaaaaasecondary10.45%1sampleof sampled distinct values–7not kept
4last_nameLULf2aAaaaaasecondary10.45%1sampleof sampled distinct values–7oBucher
4last_nameLULf2aAa{7}secondary10.45%1sampleof sampled distinct values–9oRdriguez
4last_nameLf3aaasecondary10.45%1sampleof sampled distinct values–3lee
4last_nameULP Lf4Aa aaasecondary10.45%1sampleof sampled distinct values–6Sm ith
4last_nameULP Lf4Aaa' 'aaaasecondary10.45%1sampleof sampled distinct values–9Rob erts
4last_nameULP Lf4Aaaa' 'aaaasecondary10.45%1sampleof sampled distinct values–10not kept
4last_nameULP Lf4Aaaaaa asecondary10.45%1sampleof sampled distinct values–8not kept
4last_nameULP Lf4Aaaaaaa asecondary10.45%1sampleof sampled distinct values–9not kept
4last_nameXf5@@@@@secondary10.45%1sampleof sampled distinct values0.17%1655not kept
4last_nameXf5@@@@@@secondary10.45%1sampleof sampled distinct values0.05%536not kept
4last_nameUP Lf6A aaasecondary10.45%1sampleof sampled distinct values–5not kept
4last_nameUP Lf6A aaaasecondary10.45%1sampleof sampled distinct values–6not kept
4last_nameUP Lf6A' 'aaaaasecondary10.45%1sampleof sampled distinct values–8T aylor
4last_nameUP'ULf7A''''Aaaaaasecondary10.45%1sampleof sampled distinct values0.41%4118not kept
4last_nameP'ULf9''''Aaaaasecondary10.45%1sampleof sampled distinct values–6not kept
4last_nameULP ULf10Aa Aaaasecondary10.45%1sampleof sampled distinct values0.06%557De Luca
4last_nameULP-ULf11Aa-Aaaaaasecondary10.45%1sampleof sampled distinct values0.05%499not kept
5birth_dateDP-DP-Df19999-99-99principal6767.0%67sampleof sampled distinct values72.71%72,710101976-06-271993-10-061966-04-20
5birth_dateDP-DP-Df199-99-9999secondary3333.0%33sampleof sampled distinct values27.28%27,2751027-07-195026-02-198929-02-2000
16postal_codeUDUP DUDf1A9A 9A9principal10057.14%100sampleof sampled distinct values99.92%99,9257A0A 6M2H0Y 3S5H0M 7Y2
16postal_codeUDUP DUDf1A9A' '9A9secondary5430.86%54sampleof sampled distinct values0.05%548not kept
16postal_codeUDUP DUf2A9A 9Asecondary74.0%7sampleof sampled distinct values0.01%76not kept
16postal_codeUDUDUDf3A'0'A9A9secondary52.86%5sampleof sampled distinct values0.01%56not kept
16postal_codeDUP DUDf4'0'A 9A9secondary31.71%3sampleof sampled distinct values–6not kept
16postal_codeUDP DUDf5A'0' 9A9secondary31.71%3sampleof sampled distinct values–6not kept
16postal_codeUDUP UDf6A9A A9secondary21.14%2sampleof sampled distinct values–6not kept
16postal_codeUDUP Df7A9A 99secondary10.57%1sampleof sampled distinct values–6not kept
19phoneP+DP-DP-DP-Df1'+1-2'99'-555-01'99principal10073.53%100sampleof sampled distinct values99.96%99,95815+1-205-555-0104+1-206-555-0120+1-203-555-0121
19phoneP+DP DP-DP-Df2'+1 20'9'-555-01'99secondary3626.47%36sampleof sampled distinct values0.04%4216+1 200-555-0171+1 201-555-0170+1 200-555-0167
20contract_idUP-Df1'CTR-'9{10}principal10073.53%100sampleof sampled distinct values99.96%99,95514CTR-1000051543CTR-3000010844CTR-1000032588
20contract_idUP-Df1'CTR-'9{9}secondary2216.18%22sampleof sampled distinct values0.02%2213not kept
20contract_idUP-Df1'CTR--'9{10}secondary1410.29%14sampleof sampled distinct values0.01%1415not kept
22start_dateDP-DP-Df199-99'-20'99principal6060.0%60sampleof sampled distinct values27.27%27,2751014-01-202111-18-201708-08-2020
22start_dateDP-DP-Df1'20'99-99-99secondary4040.0%40sampleof sampled distinct values72.72%72,717102017-08-212023-10-182023-07-12
23end_dateDP-DP-Df199-99'-20'99principal6161.0%61sampleof sampled distinct values27.05%23,3331003-11-202212-06-202006-05-2023
23end_dateDP-DP-Df1'20'99-99-99secondary3939.0%39sampleof sampled distinct values72.94%62,909102023-10-182027-02-152023-07-12
27payment_methodLP_Lf1aaaaaa_aaaaaprincipal111.11%1exactof every distinct value63.69%63,68812direct_debit
27payment_methodLP_Lf1aaaaaa_aaaasecondary111.11%1exactof every distinct value29.26%29,25811credit_card
27payment_methodLP_Lf1aaaa_a{8}secondary111.11%1exactof every distinct value5.27%5,26913bank_transfer
27payment_methodLf2aaaaasecondary111.11%1exactof every distinct value–5check
27payment_methodLf2aaaaaasecondary111.11%1exactof every distinct value1.76%1,7606cheque
27payment_methodLP_LP_Uf3aaaaaa_aaaa_Asecondary111.11%1exactof every distinct value–13credit_card_X
27payment_methodLP_LP_Uf3aaaaaa_aaaaa_Asecondary111.11%1exactof every distinct value0.01%514direct_debit_X
27payment_methodLP Lf4aaaaaa aaaaasecondary111.11%1exactof every distinct value0.02%1612direct debit
27payment_methodLP_Uf5aaaaaa_Asecondary111.11%1exactof every distinct value–8cheque_X
28agency_codeUP-UP-Df1'AG-'AA'-00'9principal2066.67%20exactof every distinct value99.98%99,9849AG-PR-002AG-PR-003AG-PR-004
28agency_codeUP-UP-Df1'AG--'AA'-00'9secondary1033.33%10exactof every distinct value0.02%1610AG--PR-004AG--PR-002AG--PR-001
29creation_dateDP-DP-Df199-99'-20'99principal6060.0%60sampleof sampled distinct values27.27%27,2751014-01-202108-08-202013-02-2022
29creation_dateDP-DP-Df1'20'99-99-99secondary4040.0%40sampleof sampled distinct values72.72%72,716102017-08-212023-10-182023-07-12
30modification_dateDP-DP-Df199-99'-20'99principal5959.0%59sampleof sampled distinct values27.27%27,2751014-01-202108-08-202013-02-2022
30modification_dateDP-DP-Df1'20'99-99-99secondary4040.0%40sampleof sampled distinct values72.72%72,718102017-08-212023-10-182023-07-12
30modification_dateDP/DP/Df29999/99/99secondary11.0%1sampleof sampled distinct values0.01%5102026/09/11
31claim_dateDP-DP-Df1'20'99-99-99principal5555.0%55sampleof sampled distinct values72.33%10,187102023-10-182023-07-122025-08-18
31claim_dateDP-DP-Df199-99'-20'99secondary4545.0%45sampleof sampled distinct values27.67%3,8981023-12-202003-06-202524-02-2024

A share is the part of the distinct values a pattern takes: exact when the scan held every distinct value, sample when it held a sample of them. A value counts for one pattern: the one of its family that has its lengths. Quoted text is a constant the pattern folds. Validated on every value is the second pass: how many occurrences of the whole file the pattern reads, exactly; a value no pattern reads is an observation of structure, not an error.

12 / Relationships

Relationships(8)

How one column goes with another, counted on every row by the second pass. The counts and the percentages of each category come first; a score only summarizes them. An association is not a cause, and a candidate rule is learned from the data and validated by nobody.

title to gender(1 feature)

By prefix of 2 characters exact 100,000 of 100,000 rows are classified. The dominant prefix of 2 characters of each title holds 100.0% of them; the most frequent prefix of 2 characters overall holds 50.0%.

Classified rows
100,000of 100,000
Purity
100.0%Dominant class of each category; the other way round 100.0%
Association
1.00Cramér's V, bias-corrected: strong
Exceptions
0Rows outside the dominant class, in 0 cells
title
F
M
Rows
Coverage
Purity
Ms.50,025100.0% dominant class050,025100.0%100.0%
Mr.049,975100.0% dominant class49,975100.0%100.0%
All50,02549,975100,000

Candidate rules(2)

  • candidate When title is Ms., gender reads F in 50,025 of 50,025 rows (100.0%).Learned from the data, not validated.0
  • candidate When title is Mr., gender reads M in 49,975 of 49,975 rows (100.0%).Learned from the data, not validated.0
The same pair the other way round: gender to title

gender to title(1 feature)

By prefix of 2 characters exact 100,000 of 100,000 rows are classified. The dominant prefix of 2 characters of each gender holds 100.0% of them; the most frequent prefix of 2 characters overall holds 50.0%.

Classified rows
100,000of 100,000
Purity
100.0%Dominant class of each category; the other way round 100.0%
Association
1.00Cramér's V, bias-corrected: strong
Exceptions
0Rows outside the dominant class, in 0 cells
gender
Ms
Mr
Rows
Coverage
Purity
F50,025100.0% dominant class050,025100.0%100.0%
M049,975100.0% dominant class49,975100.0%100.0%
All50,02549,975100,000

Candidate rules(2)

  • candidate When gender is F, title reads Ms in 50,025 of 50,025 rows (100.0%).Learned from the data, not validated.0
  • candidate When gender is M, title reads Mr in 49,975 of 49,975 rows (100.0%).Learned from the data, not validated.0

branch to customer_id(1 feature)

By prefix of 2 characters exact 100,000 of 100,000 rows are classified. The dominant prefix of 2 characters of each branch holds 100.0% of them; the most frequent prefix of 2 characters overall holds 72.7%.

Classified rows
100,000of 100,000
Purity
100.0%Dominant class of each category; the other way round 100.0%
Association
1.00Cramér's V, bias-corrected: strong
Exceptions
0Rows outside the dominant class, in 0 cells
branch
CL
QC
ON
Rows
Coverage
Purity
main72,727100.0% dominant class0072,727100.0%100.0%
quebec018,182100.0% dominant class018,182100.0%100.0%
ontario009,091100.0% dominant class9,091100.0%100.0%
All72,72718,1829,091100,000

Candidate rules(3)

  • candidate When branch is main, customer_id reads CL in 72,727 of 72,727 rows (100.0%).Learned from the data, not validated.0
  • candidate When branch is quebec, customer_id reads QC in 18,182 of 18,182 rows (100.0%).Learned from the data, not validated.0
  • candidate When branch is ontario, customer_id reads ON in 9,091 of 9,091 rows (100.0%).Learned from the data, not validated.0

province_code to province_name(1 feature)

By length exact 99,997 of 100,000 rows are classified. The dominant length of each province_code holds 100.0% of them; the most frequent length overall holds 75.2%.

Classified rows
99,997of 100,000; 3 have no province_code
Purity
100.0%Dominant class of each category; the other way round 97.2%
Association
1.00Unstable: low counts
Exceptions
3Rows outside the dominant class, in 1 cells

The expected counts are below 5 in 48% of the cells: this score is not reliable and says little.

province_code
5
6
7
8
11
12
13
16
20
21
25
Rows
Coverage
Purity
QC075,173100.0% dominant class00000000075,173100.0%100.0%
ON030.0%14,327100.0% dominant class0000000014,330100.0%100.0%
AB002,604100.0% dominant class000000002,604100.0%100.0%
BC00000002,541100.0% dominant class0002,541100.0%100.0%
NB0000001,750100.0% dominant class00001,750100.0%100.0%
SK00000909100.0% dominant class00000909100.0%100.0%
MB000879100.0% dominant class0000000879100.0%100.0%
NS0000874100.0% dominant class000000874100.0%100.0%
NL0000000000361100.0% dominant class361100.0%100.0%
PE00000000188100.0% dominant class00188100.0%100.0%
NU00184100.0% dominant class00000000184100.0%100.0%
YT95100.0% dominant class000000000095100.0%100.0%
NT00000000086100.0% dominant class086100.0%100.0%
qc017100.0% dominant class00000000017100.0%100.0%
on003100.0% dominant class000000003100.0%100.0%
ab001100.0% dominant class000000001100.0%100.0%
bc00000001100.0% dominant class0001100.0%100.0%
nt0000000001100.0% dominant class01100.0%100.0%
All9575,19317,1198798749091,7502,5421888736199,997

Exceptions(3 rows)

province_code
Class
Rows
First rows
ON63QuebecQuebecQuebecRecords 41034, 59460, 84385

Candidate rules(13)

  • candidate When province_code is QC, province_name reads 6 in 75,173 of 75,173 rows (100.0%).Learned from the data, not validated.0
  • candidate When province_code is ON, province_name reads 7 in 14,327 of 14,330 rows (100.0%).Learned from the data, not validated.3
  • candidate When province_code is AB, province_name reads 7 in 2,604 of 2,604 rows (100.0%).Learned from the data, not validated.0
  • candidate When province_code is BC, province_name reads 16 in 2,541 of 2,541 rows (100.0%).Learned from the data, not validated.0
  • candidate When province_code is NB, province_name reads 13 in 1,750 of 1,750 rows (100.0%).Learned from the data, not validated.0
  • candidate When province_code is SK, province_name reads 12 in 909 of 909 rows (100.0%).Learned from the data, not validated.0
  • candidate When province_code is MB, province_name reads 8 in 879 of 879 rows (100.0%).Learned from the data, not validated.0
  • candidate When province_code is NS, province_name reads 11 in 874 of 874 rows (100.0%).Learned from the data, not validated.0
  • candidate When province_code is NL, province_name reads 25 in 361 of 361 rows (100.0%).Learned from the data, not validated.0
  • candidate When province_code is PE, province_name reads 20 in 188 of 188 rows (100.0%).Learned from the data, not validated.0
  • candidate When province_code is NU, province_name reads 7 in 184 of 184 rows (100.0%).Learned from the data, not validated.0
  • candidate When province_code is YT, province_name reads 5 in 95 of 95 rows (100.0%).Learned from the data, not validated.0
  • candidate When province_code is NT, province_name reads 21 in 86 of 86 rows (100.0%).Learned from the data, not validated.0
The same pair the other way round: province_name to province_code

province_name to province_code(1 feature)

By prefix of 2 characters exact 99,997 of 100,000 rows are classified. The dominant prefix of 2 characters of each province_name holds 100.0% of them; the most frequent prefix of 2 characters overall holds 75.2%.

Classified rows
99,997of 100,000; 3 have no value in province_code
Purity
100.0%Dominant class of each category; the other way round 99.9%
Association
0.99Unstable: low counts
Exceptions
25Rows outside the dominant class, in 5 cells

The expected counts are below 5 in 46% of the cells: this score is not reliable and says little.

province_name
QC
ON
AB
BC
NB
SK
MB
NS
NL
PE
NU
other
Rows
Coverage
Purity
Quebec75,163100.0% dominant class30.0%000000000170.0%75,183100.0%100.0%
Ontario014,326100.0% dominant class00000000030.0%14,329100.0%100.0%
Alberta002,604100.0% dominant class0000000010.0%2,605100.0%100.0%
British Columbia0002,540100.0% dominant class000000010.0%2,541100.0%100.0%
New Brunswick00001,750100.0% dominant class00000001,750100.0%100.0%
Saskatchewan00000909100.0% dominant class000000909100.0%100.0%
Manitoba000000879100.0% dominant class00000879100.0%100.0%
Nova Scotia0000000874100.0% dominant class0000874100.0%100.0%
Newfoundland and Labrador00000000361100.0% dominant class000361100.0%100.0%
Prince Edward Island000000000188100.0% dominant class00188100.0%100.0%
Nunavut0000000000184100.0% dominant class0184100.0%100.0%
Yukon0000000000095100.0% dominant class95100.0%100.0%
Northwest Territories0000000000087100.0% dominant class87100.0%100.0%
quebec10100.0% dominant class0000000000010100.0%100.0%
british columbia0001100.0% dominant class000000001100.0%100.0%
ontario01100.0% dominant class00000000001100.0%100.0%
All75,17314,3302,6042,5411,75090987987436118818420499,997

Exceptions(25 rows)

province_name
Class
Rows
First rows
Quebecother17qcqcqcRecords 335, 1436, 15848
QuebecON3ONONONRecords 41034, 59460, 84385
Ontarioother3onononRecords 21795, 75950, 81759
Albertaother1abRecords 16556
British Columbiaother1bcRecords 6691

Candidate rules(11)

  • candidate When province_name is Quebec, province_code reads QC in 75,163 of 75,183 rows (100.0%).Learned from the data, not validated.20
  • candidate When province_name is Ontario, province_code reads ON in 14,326 of 14,329 rows (100.0%).Learned from the data, not validated.3
  • candidate When province_name is Alberta, province_code reads AB in 2,604 of 2,605 rows (100.0%).Learned from the data, not validated.1
  • candidate When province_name is British Columbia, province_code reads BC in 2,540 of 2,541 rows (100.0%).Learned from the data, not validated.1
  • candidate When province_name is New Brunswick, province_code reads NB in 1,750 of 1,750 rows (100.0%).Learned from the data, not validated.0
  • candidate When province_name is Saskatchewan, province_code reads SK in 909 of 909 rows (100.0%).Learned from the data, not validated.0
  • candidate When province_name is Manitoba, province_code reads MB in 879 of 879 rows (100.0%).Learned from the data, not validated.0
  • candidate When province_name is Nova Scotia, province_code reads NS in 874 of 874 rows (100.0%).Learned from the data, not validated.0
  • candidate When province_name is Newfoundland and Labrador, province_code reads NL in 361 of 361 rows (100.0%).Learned from the data, not validated.0
  • candidate When province_name is Prince Edward Island, province_code reads PE in 188 of 188 rows (100.0%).Learned from the data, not validated.0
  • candidate When province_name is Nunavut, province_code reads NU in 184 of 184 rows (100.0%).Learned from the data, not validated.0

agency_code to customer_id(1 feature)

By prefix of 2 characters exact 100,000 of 100,000 rows are classified. The dominant prefix of 2 characters of each agency_code holds 100.0% of them; the most frequent prefix of 2 characters overall holds 72.7%.

Classified rows
100,000of 100,000
Purity
100.0%Dominant class of each category; the other way round 36.5%
Association
1.00Cramér's V, bias-corrected: strong
Exceptions
0Rows outside the dominant class, in 0 cells
agency_code
CL
QC
ON
Rows
Coverage
Purity
AG-PR-00127,096100.0% dominant class0027,096100.0%100.0%
AG-PR-00216,829100.0% dominant class0016,829100.0%100.0%
AG-PR-0039,094100.0% dominant class009,094100.0%100.0%
AG-PR-0048,100100.0% dominant class008,100100.0%100.0%
AG-QC-00106,334100.0% dominant class06,334100.0%100.0%
AG-QC-00204,000100.0% dominant class04,000100.0%100.0%
AG-PR-0053,899100.0% dominant class003,899100.0%100.0%
AG-ON-001003,088100.0% dominant class3,088100.0%100.0%
AG-PR-0062,792100.0% dominant class002,792100.0%100.0%
AG-QC-00302,561100.0% dominant class02,561100.0%100.0%
AG-PR-0082,515100.0% dominant class002,515100.0%100.0%
AG-PR-0072,394100.0% dominant class002,394100.0%100.0%
AG-QC-00402,152100.0% dominant class02,152100.0%100.0%
AG-ON-002001,812100.0% dominant class1,812100.0%100.0%
AG-QC-00501,786100.0% dominant class01,786100.0%100.0%
AG-ON-003001,452100.0% dominant class1,452100.0%100.0%
AG-QC-00601,343100.0% dominant class01,343100.0%100.0%
AG-ON-004001,137100.0% dominant class1,137100.0%100.0%
AG-ON-00500989100.0% dominant class989100.0%100.0%
AG-ON-00600611100.0% dominant class611100.0%100.0%
AG--PR-0023100.0% dominant class003100.0%100.0%
AG--QC-00103100.0% dominant class03100.0%100.0%
AG--ON-003002100.0% dominant class2100.0%100.0%
AG--PR-0042100.0% dominant class002100.0%100.0%
AG--PR-0011100.0% dominant class001100.0%100.0%
AG--PR-0031100.0% dominant class001100.0%100.0%
AG--PR-0071100.0% dominant class001100.0%100.0%
AG--QC-00201100.0% dominant class01100.0%100.0%
AG--QC-00401100.0% dominant class01100.0%100.0%
AG--QC-00601100.0% dominant class01100.0%100.0%
All72,72718,1829,091100,000

Candidate rules(20)

  • candidate When agency_code is AG-PR-001, customer_id reads CL in 27,096 of 27,096 rows (100.0%).Learned from the data, not validated.0
  • candidate When agency_code is AG-PR-002, customer_id reads CL in 16,829 of 16,829 rows (100.0%).Learned from the data, not validated.0
  • candidate When agency_code is AG-PR-003, customer_id reads CL in 9,094 of 9,094 rows (100.0%).Learned from the data, not validated.0
  • candidate When agency_code is AG-PR-004, customer_id reads CL in 8,100 of 8,100 rows (100.0%).Learned from the data, not validated.0
  • candidate When agency_code is AG-QC-001, customer_id reads QC in 6,334 of 6,334 rows (100.0%).Learned from the data, not validated.0
  • candidate When agency_code is AG-QC-002, customer_id reads QC in 4,000 of 4,000 rows (100.0%).Learned from the data, not validated.0
  • candidate When agency_code is AG-PR-005, customer_id reads CL in 3,899 of 3,899 rows (100.0%).Learned from the data, not validated.0
  • candidate When agency_code is AG-ON-001, customer_id reads ON in 3,088 of 3,088 rows (100.0%).Learned from the data, not validated.0
  • candidate When agency_code is AG-PR-006, customer_id reads CL in 2,792 of 2,792 rows (100.0%).Learned from the data, not validated.0
  • candidate When agency_code is AG-QC-003, customer_id reads QC in 2,561 of 2,561 rows (100.0%).Learned from the data, not validated.0
  • candidate When agency_code is AG-PR-008, customer_id reads CL in 2,515 of 2,515 rows (100.0%).Learned from the data, not validated.0
  • candidate When agency_code is AG-PR-007, customer_id reads CL in 2,394 of 2,394 rows (100.0%).Learned from the data, not validated.0
  • candidate When agency_code is AG-QC-004, customer_id reads QC in 2,152 of 2,152 rows (100.0%).Learned from the data, not validated.0
  • candidate When agency_code is AG-ON-002, customer_id reads ON in 1,812 of 1,812 rows (100.0%).Learned from the data, not validated.0
  • candidate When agency_code is AG-QC-005, customer_id reads QC in 1,786 of 1,786 rows (100.0%).Learned from the data, not validated.0
  • candidate When agency_code is AG-ON-003, customer_id reads ON in 1,452 of 1,452 rows (100.0%).Learned from the data, not validated.0
  • candidate When agency_code is AG-QC-006, customer_id reads QC in 1,343 of 1,343 rows (100.0%).Learned from the data, not validated.0
  • candidate When agency_code is AG-ON-004, customer_id reads ON in 1,137 of 1,137 rows (100.0%).Learned from the data, not validated.0
  • candidate When agency_code is AG-ON-005, customer_id reads ON in 989 of 989 rows (100.0%).Learned from the data, not validated.0
  • candidate When agency_code is AG-ON-006, customer_id reads ON in 611 of 611 rows (100.0%).Learned from the data, not validated.0

agency_code to branch(1 feature)

By length exact 100,000 of 100,000 rows are classified. The dominant length of each agency_code holds 100.0% of them; the most frequent length overall holds 72.7%.

Classified rows
100,000of 100,000
Purity
100.0%Dominant class of each category; the other way round 36.5%
Association
1.00Cramér's V, bias-corrected: strong
Exceptions
0Rows outside the dominant class, in 0 cells
agency_code
4
6
7
Rows
Coverage
Purity
AG-PR-00127,096100.0% dominant class0027,096100.0%100.0%
AG-PR-00216,829100.0% dominant class0016,829100.0%100.0%
AG-PR-0039,094100.0% dominant class009,094100.0%100.0%
AG-PR-0048,100100.0% dominant class008,100100.0%100.0%
AG-QC-00106,334100.0% dominant class06,334100.0%100.0%
AG-QC-00204,000100.0% dominant class04,000100.0%100.0%
AG-PR-0053,899100.0% dominant class003,899100.0%100.0%
AG-ON-001003,088100.0% dominant class3,088100.0%100.0%
AG-PR-0062,792100.0% dominant class002,792100.0%100.0%
AG-QC-00302,561100.0% dominant class02,561100.0%100.0%
AG-PR-0082,515100.0% dominant class002,515100.0%100.0%
AG-PR-0072,394100.0% dominant class002,394100.0%100.0%
AG-QC-00402,152100.0% dominant class02,152100.0%100.0%
AG-ON-002001,812100.0% dominant class1,812100.0%100.0%
AG-QC-00501,786100.0% dominant class01,786100.0%100.0%
AG-ON-003001,452100.0% dominant class1,452100.0%100.0%
AG-QC-00601,343100.0% dominant class01,343100.0%100.0%
AG-ON-004001,137100.0% dominant class1,137100.0%100.0%
AG-ON-00500989100.0% dominant class989100.0%100.0%
AG-ON-00600611100.0% dominant class611100.0%100.0%
AG--PR-0023100.0% dominant class003100.0%100.0%
AG--QC-00103100.0% dominant class03100.0%100.0%
AG--ON-003002100.0% dominant class2100.0%100.0%
AG--PR-0042100.0% dominant class002100.0%100.0%
AG--PR-0011100.0% dominant class001100.0%100.0%
AG--PR-0031100.0% dominant class001100.0%100.0%
AG--PR-0071100.0% dominant class001100.0%100.0%
AG--QC-00201100.0% dominant class01100.0%100.0%
AG--QC-00401100.0% dominant class01100.0%100.0%
AG--QC-00601100.0% dominant class01100.0%100.0%
All72,72718,1829,091100,000

Candidate rules(20)

  • candidate When agency_code is AG-PR-001, branch reads 4 in 27,096 of 27,096 rows (100.0%).Learned from the data, not validated.0
  • candidate When agency_code is AG-PR-002, branch reads 4 in 16,829 of 16,829 rows (100.0%).Learned from the data, not validated.0
  • candidate When agency_code is AG-PR-003, branch reads 4 in 9,094 of 9,094 rows (100.0%).Learned from the data, not validated.0
  • candidate When agency_code is AG-PR-004, branch reads 4 in 8,100 of 8,100 rows (100.0%).Learned from the data, not validated.0
  • candidate When agency_code is AG-QC-001, branch reads 6 in 6,334 of 6,334 rows (100.0%).Learned from the data, not validated.0
  • candidate When agency_code is AG-QC-002, branch reads 6 in 4,000 of 4,000 rows (100.0%).Learned from the data, not validated.0
  • candidate When agency_code is AG-PR-005, branch reads 4 in 3,899 of 3,899 rows (100.0%).Learned from the data, not validated.0
  • candidate When agency_code is AG-ON-001, branch reads 7 in 3,088 of 3,088 rows (100.0%).Learned from the data, not validated.0
  • candidate When agency_code is AG-PR-006, branch reads 4 in 2,792 of 2,792 rows (100.0%).Learned from the data, not validated.0
  • candidate When agency_code is AG-QC-003, branch reads 6 in 2,561 of 2,561 rows (100.0%).Learned from the data, not validated.0
  • candidate When agency_code is AG-PR-008, branch reads 4 in 2,515 of 2,515 rows (100.0%).Learned from the data, not validated.0
  • candidate When agency_code is AG-PR-007, branch reads 4 in 2,394 of 2,394 rows (100.0%).Learned from the data, not validated.0
  • candidate When agency_code is AG-QC-004, branch reads 6 in 2,152 of 2,152 rows (100.0%).Learned from the data, not validated.0
  • candidate When agency_code is AG-ON-002, branch reads 7 in 1,812 of 1,812 rows (100.0%).Learned from the data, not validated.0
  • candidate When agency_code is AG-QC-005, branch reads 6 in 1,786 of 1,786 rows (100.0%).Learned from the data, not validated.0
  • candidate When agency_code is AG-ON-003, branch reads 7 in 1,452 of 1,452 rows (100.0%).Learned from the data, not validated.0
  • candidate When agency_code is AG-QC-006, branch reads 6 in 1,343 of 1,343 rows (100.0%).Learned from the data, not validated.0
  • candidate When agency_code is AG-ON-004, branch reads 7 in 1,137 of 1,137 rows (100.0%).Learned from the data, not validated.0
  • candidate When agency_code is AG-ON-005, branch reads 7 in 989 of 989 rows (100.0%).Learned from the data, not validated.0
  • candidate When agency_code is AG-ON-006, branch reads 7 in 611 of 611 rows (100.0%).Learned from the data, not validated.0

province_code to postal_code(1 feature)

By prefix of 2 characters exact 99,997 of 100,000 rows are classified. The dominant prefix of 2 characters of each province_code holds 85.3% of them; the most frequent prefix of 2 characters overall holds 60.5%.

Classified rows
99,997of 100,000; 3 have no province_code
Purity
85.3%Dominant class of each category; the other way round 97.7%
Association
0.93Unstable: low counts
Exceptions
14,686Rows outside the dominant class, in 4 cells

The expected counts are below 5 in 27% of the cells: this score is not reliable and says little.

province_code
H0
H1
K0
T0
V0
A0
S0
other
Rows
Coverage
Purity
QC60,49280.5% dominant class14,67819.5%0000030.0%75,173100.0%80.5%
ON30.0%014,327100.0% dominant class0000014,330100.0%100.0%
AB0002,604100.0% dominant class00002,604100.0%100.0%
BC00002,541100.0% dominant class0002,541100.0%100.0%
NB00000001,750100.0% dominant class1,750100.0%100.0%
SK000000909100.0% dominant class0909100.0%100.0%
MB0000000879100.0% dominant class879100.0%100.0%
NS0000000874100.0% dominant class874100.0%100.0%
NL00000361100.0% dominant class00361100.0%100.0%
PE0000000188100.0% dominant class188100.0%100.0%
NU0000000184100.0% dominant class184100.0%100.0%
YT000000095100.0% dominant class95100.0%100.0%
NT000000086100.0% dominant class86100.0%100.0%
qc1588.2% dominant class211.8%00000017100.0%88.2%
on003100.0% dominant class000003100.0%100.0%
ab0001100.0% dominant class00001100.0%100.0%
bc00001100.0% dominant class0001100.0%100.0%
nt00000001100.0% dominant class1100.0%100.0%
All60,51014,68014,3302,6052,5423619094,06099,997

Exceptions(14,686 rows)

province_code
Class
Rows
First rows
QCH114,678H1N 9R6H1P 7K5H1B 7G8Records 5, 21, 26
QCother30Z 3L60G 7K90E 0E7Records 24617, 31653, 63471
ONH03H0A 7Y1H0A 7T4H0A 7Y2Records 41034, 59460, 84385
qcH12H1M 0B2H1R 5B1Records 28142, 63900

Candidate rules(5)

  • candidate When province_code is ON, postal_code reads K0 in 14,327 of 14,330 rows (100.0%).Learned from the data, not validated.3
  • candidate When province_code is AB, postal_code reads T0 in 2,604 of 2,604 rows (100.0%).Learned from the data, not validated.0
  • candidate When province_code is BC, postal_code reads V0 in 2,541 of 2,541 rows (100.0%).Learned from the data, not validated.0
  • candidate When province_code is SK, postal_code reads S0 in 909 of 909 rows (100.0%).Learned from the data, not validated.0
  • candidate When province_code is NL, postal_code reads A0 in 361 of 361 rows (100.0%).Learned from the data, not validated.0

779 tables of a category against a feature were explored on the sampled rows of the interior (900 rows): 20 listed, 61 only one category has rows, 93 only one class has rows. A relation not found in a sample may still exist in the file.

6 pairs of numeric columns were explored on the sampled rows of the interior (900 rows): 0 listed.

Counts are exact: every row of the file is placed in one category and one class, or counted as left out. The dominant class of a category is the one that holds most of it; a tie goes to the first class listed. Purity is the share of the classified rows that sit in the dominant class of their category. The score is Cramér's V corrected for the number of rows and classes; when the expected counts are low it is marked unstable and given no strength. An empty other row or column is not printed. Correlations show a linear association between two columns, over the rows where both have a number.

13 / Insights

Insights(11)

What the second pass found in the values, with the accuracy of each figure: exact counts every value of the file, estimated comes with certain bounds. The same cards appear on the page of each column they concern.

Values left out

Values left out by patterns and detectorsfirst_name

exact

99,993 values read.

Values read
99,993

Each pattern over every occurrence exact

Pattern
Occurrences
Distinct values
Example
First record
Aaaaaa24,97227Hudson2
Aaaa23,77322Emma1
Aaaaa22,69129Elias6
Aa{7}8,57311Benjamin3
Aaaaaaa8,10411William42
Aa{8}6,3344Elizabeth11
Aaa5,5184Ava4
AAaaaaa33OOliver14,498
Aa{9}11Elizaabeth80,588
aaaa44acob24,511
aaaaa22ethan72,963
a{7}22william31,215
aaa11mia35,625
aaaaaa11harper21,204
Aaaa' 'a22Davi d11,575

Values left out by patterns and detectorslast_name

exact

99,996 values read.

Values read
99,996

Each pattern over every occurrence exact

Pattern
Occurrences
Distinct values
Example
First record
Aaaaa32,12032Clark1
Aaaaaa25,49443Nelson2
Aa{7}15,75128Anderson19
Aaaa10,29319Ward10
Aaaaaaa8,04519Johnson11
Aaa2,8448Cox69
Aa{8}1,5886Pelletier8
Aa1,3924Wu110
Aa{9}6723Richardson39
AAaaaa11PPatel37,143
AAaaaaa11WWright75,530
aAaaa65hAmed1,766
aAaaaaaa85nAderson10,720
aAaaaa44oHward17,292
aAaa33aHll8,220

Values left out by patterns and detectorsphone

exact

100,000 values read.

Values read
100,000

Each pattern over every occurrence exact

Pattern
Occurrences
Distinct values
Example
First record
'+1-2'99'-555-01'9999,9581,181+1-200-555-01331
'+1 20'9'-555-01'994236+1 200-555-01355,491

Values left out by patterns and detectorsstart_date

exact

100,000 values read.

Values read
100,000
Left out: no pattern reads them
8 occurrences8 distinct values

Most frequent values left out (1) exact

Value
Occurrences
First record
11/07/2024187,425
2017/12/23153,226
2019/03/30114,696
2020/04/15111,283
2020/12/06155,526
2025/10/29170,635
2026/07/09143,904
29/03/2025182,374

Each pattern over every occurrence exact

Pattern
Occurrences
Distinct values
Example
First record
'20'99-99-9972,7173,6612020-01-011
99-99'-20'9927,2755,64613-11-2018442

Values left out by patterns and detectorsend_date

exact

86,250 values read.

Values read
86,250
Left out: no pattern reads them
8 occurrences8 distinct values

Most frequent values left out (1) exact

Value
Occurrences
First record
04/08/2027193,088
07/25/2027177,822
10/03/2027183,860
2023/12/10116,698
2026/09/13114,780
2026/10/21114,957
2027/03/30120,469
2027/05/10130,659

Each pattern over every occurrence exact

Pattern
Occurrences
Distinct values
Example
First record
99-99'-20'9923,3333,25021-11-202621,689
'20'99-99-9962,9092,9902027-02-121

Values left out by patterns and detectorscreation_date

exact

100,000 values read.

Values read
100,000
Left out: no pattern reads them
9 occurrences9 distinct values

Most frequent values left out (1) exact

Value
Occurrences
First record
03/05/2019176,213
06/20/2023179,701
2017/07/11126,585
2019/03/1418,973
2020/05/16163,239
2020/05/19140,466
2022/07/09163,454
2023/10/121329
2025/08/25167,818

Each pattern over every occurrence exact

Pattern
Occurrences
Distinct values
Example
First record
'20'99-99-9972,7163,6782019-12-311
99-99'-20'9927,2755,66822-12-20171,162

Values left out by patterns and detectorsmodification_date

exact

100,000 values read.

Values read
100,000
Left out: no pattern reads them
2 occurrences2 distinct values

Most frequent values left out (1) exact

Value
Occurrences
First record
03/03/2023185,522
30/12/2025189,153

Each pattern over every occurrence exact

Pattern
Occurrences
Distinct values
Example
First record
'20'99-99-9972,7183,3712022-02-021
99-99'-20'9927,2754,38120-03-20261,492
9999/99/99552024/10/1812,037

Values left out by patterns and detectorsclaim_date

exact

14,085 values read.

Values read
14,085

Each pattern over every occurrence exact

Pattern
Occurrences
Distinct values
Example
First record
'20'99-99-9910,1872,6342025-05-123
99-99'-20'993,8982,24007-25-201872,730

Repeated values

Repeated valuescustomer_id

exact

8 values found more than once.

Values read
100,000
Distinct values
99,992
Values found more than once
8
Extra occurrences
8

Most repeated values exact

Value
Occurrences
First record
CLI-000006962696
CLI-0000685526,855
CLI-00024687224,687
CLI-00047806247,806
CLI-00054365254,365
CLI-00069732269,732
QC-0000-1479283,297
QC-0000-7867289,685

Repeated valuespostal_code

exact

22,241 values found more than once.

Values read
100,000
Distinct values
57,542
Values found more than once
22,241
Extra occurrences
42,458

Most repeated values exact

Value
Occurrences
First record
H0A 7S5154,646
H0A 8T3154,094
H0A 3H9143,095
H0A 1T0135,328
H0A 2V8134,878
H0A 7L2134,571
H0A 7W6138,844
H0A 8A3131,970
H0A 8R11313,007
H0A 0G21212,310
H0A 1N1124,399
H0A 1V8123,950
H0A 3C4123,157
H0A 3R8122,777
H0A 4Z61234,912

The list is bounded; the counts of the values it holds are exact.

Numbers by category

Numbers by categorycontract_type and annual_premium

exactestimated

99,981 of 100,000 rows.

Rows
100,000
Rows with a category and a number
99,981
Rows without a category
2
Rows without a number
17
Share of the variance the categories explain
33.1%33.0% once corrected for the number of categories

contract_type and annual_premium by category estimated

Category
Rows
Numbers
Minimum
Median
Mean
Maximum
auto40,79340,790-3,849.711,508.63841,561.98254,679.75
home30,11430,106-3,349.451,875.08881,874.37553,499.99
travel16,89816,895-726.93466.1267482.47661,199.46
life12,18212,179-10,1522,969.48553,453.718210,916.49
AUTO332,020.792,278.40912,279.99332,670.14
travel_X33148.64592.9299661.29671,162.53
auto_X222,032.692,083.98212,100.242,167.79
autos22526.97776.4249749.235971.5
home_X11931.62931.62931.62931.62

An association between two columns, not a cause. Counts are exact; the minimum, maximum, mean and spread are computed in float64; medians are estimates between certain bounds.

14 / Second pass

Second pass(48)

Exact analyses added by reading the source again. Automatic: the planner chose what is useful and cheap. The second pass computed analyses.

Computed
48Analyses with a result
Not computed
00 skipped, 0 failed
Already answered
125By the scan, no reading needed
Not selected
161Left out by the planner
48 of 48 analyses
Column
Analysis
Status
Result
Accuracy
first_nameExact frequenciescomplete136 values counted, the most frequent 2,164 timesexact
last_nameExact frequenciescomplete220 values counted, the most frequent 4,922 timesexact
street_nameExact frequenciescomplete123 values counted, the most frequent 4,016 timesexact
unit_numberExact frequenciescomplete999 values counted, the most frequent 54 timesexact
cityExact frequenciescomplete139 values counted, the most frequent 11,183 timesexact
street_numberHistogram and exact quantilescomplete9,999 distinct integers counted one by one; median 5,006exact
unit_numberHistogram and exact quantilescomplete999 distinct integers counted one by one; median 499exact
annual_premiumHistogram and exact quantilescomplete100 bins, counts exact; median estimated at 1,440.2245 (between 1,435.6695 and 1,646.3544)exact
claim_amountHistogram and exact quantilescomplete100 bins, counts exact; median estimated at 4,662.2509 (between 909.727 and 5,502.526)exact
customer_idPattern validationcomplete100,000 of 100,000 values read by 9 patterns, 0 by noneexact
titlePattern validationcomplete100,000 of 100,000 values read by 1 pattern, 0 by noneexact
first_namePattern validationcomplete99,993 of 99,993 values read by 26 patterns, 0 by noneexact
last_namePattern validationcomplete99,996 of 99,996 values read by 41 patterns, 0 by noneexact
birth_datePattern validationcomplete99,985 of 99,994 values read by 2 patterns, 9 by noneexact
postal_codePattern validationcomplete100,000 of 100,000 values read by 8 patterns, 0 by noneexact
phonePattern validationcomplete100,000 of 100,000 values read by 2 patterns, 0 by noneexact
contract_idPattern validationcomplete99,991 of 99,991 values read by 3 patterns, 0 by noneexact
start_datePattern validationcomplete99,992 of 100,000 values read by 2 patterns, 8 by noneexact
end_datePattern validationcomplete86,242 of 86,250 values read by 2 patterns, 8 by noneexact
payment_methodPattern validationcomplete100,000 of 100,000 values read by 9 patterns, 0 by noneexact
agency_codePattern validationcomplete100,000 of 100,000 values read by 2 patterns, 0 by noneexact
creation_datePattern validationcomplete99,991 of 100,000 values read by 2 patterns, 9 by noneexact
modification_datePattern validationcomplete100,000 of 100,000 values read by 3 patterns, 2 by noneexact
claim_datePattern validationcomplete14,085 of 14,085 values read by 2 patterns, 0 by noneexact
start_dateDistribution over timecomplete90,138 dates by month, 2016-09 to 2027-08; 9,862 ambiguous not placedexact
creation_dateDistribution over timecomplete90,100 dates by month, 2016-08 to 2026-09; 9,900 ambiguous not placedexact
modification_dateDistribution over timecomplete90,098 dates by month, 2016-08 to 2026-09; 9,902 ambiguous not placedexact
claim_dateDistribution over timecomplete12,646 dates by month, 2016-10 to 2026-09; 1,439 ambiguous not placedexact
first_nameValues left out by patterns and detectorscompleteat least 0 values left out; 26 patterns profiledexact
last_nameValues left out by patterns and detectorscompleteat least 0 values left out; 41 patterns profiledexact
phoneValues left out by patterns and detectorscompleteat least 0 values left out; 2 patterns profiledexact
start_dateValues left out by patterns and detectorscompleteat least 8 values left out (1 table); 2 patterns profiledexact
end_dateValues left out by patterns and detectorscompleteat least 8 values left out (1 table); 2 patterns profiledexact
creation_dateValues left out by patterns and detectorscompleteat least 9 values left out (1 table); 2 patterns profiledexact
modification_dateValues left out by patterns and detectorscompleteat least 2 values left out (1 table); 3 patterns profiledexact
claim_dateValues left out by patterns and detectorscompleteat least 0 values left out; 2 patterns profiledexact
customer_idRepeated valuescomplete8 values found more than once, 8 extra occurrencesexact
postal_codeRepeated valuescomplete22,241 values found more than once, 42,458 extra occurrences; list boundedexact
title and gender (prefix)Category against a feature of a columncomplete100,000 of 100,000 rows by prefix; Cramér's V 1.00; purity 100.0%; 2 candidate rulesexact
gender and title (prefix)Category against a feature of a columncomplete100,000 of 100,000 rows by prefix; Cramér's V 1.00; purity 100.0%; 2 candidate rulesexact
branch and customer_id (prefix)Category against a feature of a columncomplete100,000 of 100,000 rows by prefix; Cramér's V 1.00; purity 100.0%; 3 candidate rulesexact
province_name and province_code (prefix)Category against a feature of a columncomplete99,997 of 100,000 rows by prefix; Cramér's V 1.00 (low counts); purity 100.0%; 11 candidate rulesexact
province_code and province_name (length)Category against a feature of a columncomplete99,997 of 100,000 rows by length; Cramér's V 1.00 (low counts); purity 100.0%; 13 candidate rulesexact
agency_code and customer_id (prefix)Category against a feature of a columncomplete100,000 of 100,000 rows by prefix; Cramér's V 1.00; purity 100.0%; 20 candidate rulesexact
agency_code and branch (length)Category against a feature of a columncomplete100,000 of 100,000 rows by length; Cramér's V 1.00; purity 100.0%; 20 candidate rulesexact
province_code and postal_code (prefix)Category against a feature of a columncomplete99,997 of 100,000 rows by prefix; Cramér's V 0.93 (low counts); purity 85.3%; 5 candidate rulesexact
contract_type and annual_premiumNumbers of a column by categorycomplete99,981 of 100,000 rows described by 9 categories; the categories explain 33.1% of the variance (33.0% once corrected)exact
$Extra copies of the rows without an IDcompletecolumn_20: 0 extra copies among 9 rows without a valueexact

Not selected, by reason: 88 too little expected benefit for an automatic choice; 19 its CPU class is above the automatic limit; 14 too many distinct values for a table of frequencies; 14 a pattern would describe its values poorly, so the scan did not look for one; 10 the field misses no value and has too many distinct values to follow by position; 6 enough tables were already chosen automatically; 6 another feature of the same two columns says more; 2 its memory does not fit in the budget with the analyses already selected; 1 too few distinct values to infer patterns; 1 fewer than half of the values carry a currency marker, a percent sign or a unit.

The source was read again and checked against the digest of the scan, so no result mixes two versions of the data, 5.83 s. This run computed 48 analyses and reused 0 stored ones.

15 / Limits

Limits and diagnostics

What the scan could not measure completely, and the technical events it recorded. Counters and statistics continue past a limit.

Limited measures(7)

7 of 7 measures
Field
Measure
Reason
Limit
At least
last_nameInferred patternsderived.patternsmax_depthscan.pattern_inference.max_depth3–
birth_dateInferred patternsderived.patternsmax_depthscan.pattern_inference.max_depth3–
start_dateInferred patternsderived.patternsmax_depthscan.pattern_inference.max_depth3–
end_dateInferred patternsderived.patternsmax_depthscan.pattern_inference.max_depth3–
creation_dateInferred patternsderived.patternsmax_depthscan.pattern_inference.max_depth3–
modification_dateInferred patternsderived.patternsmax_depthscan.pattern_inference.max_depth3–
claim_dateInferred patternsderived.patternsmax_depthscan.pattern_inference.max_depth3–

16 / Raw values

Data sample(30)

Rows of the scan's row sample, with original row numbers and raw values: the first records, records drawn at random from the interior and the last records. The sampled rows are not uniform over the whole file.

30 of the 1,000 rows the scan sampled (10 at the start, 10 sampled, 10 at the end) out of 100,000 records. The sampled rows come from the interior, records 51 to 99,950, where each record had the same chance by a seeded hash.

30 of 30 sample rows
Row
Part
customer_idtext
titletext
first_nametext
last_nametext
birth_datedate
gendertext
preferred_languagetext
street_numberinteger
street_typetext
street_nametext
street_directiontext
unit_numberinteger
citytext
province_codetext
province_nametext
postal_codetext
country_codetext
emailtext
phonetext
contract_idtext
contract_typetext
start_datedate
end_datedate
contract_statustext
annual_premiumnumber
payment_frequencytext
payment_methodtext
agency_codetext
creation_datedate
modification_datedate
claim_datedate
claim_amountnumber
claim_statustext
branchtext
1StartCLI-00000001Ms.EmmaClark1977-08-19Fen5612TrailWestwoodemptyemptyQuébecQCQuebecH0B 8Z6CAemma.clark@outlook.example+1-200-555-0133CTR-1000000001home2020-01-012027-02-12active1072.61annualbank_transferAG-PR-0022019-12-312022-02-02emptyemptyemptymain
2StartCLI-00000002Mr.HudsonNelson1984-11-13Men6985StreetPineWestemptyGatineauQCQuebecH0E 4Y4CAhudson.nelson@gmail.example+1-200-555-0185CTR-1000000002home2021-12-072027-02-22active2141.16quarterlydirect_debitAG-PR-0032021-11-252024-06-12emptyemptyemptymain
3StartCLI-00000003Mr.BenjaminYoung1941-03-17Men4180DriveSilverwoodEastemptyGranbyQCQuebecH0V 9M7CAempty+1-200-555-0140CTR-1000000003auto2024-09-162027-03-11active292.15monthlydirect_debitAG-PR-0022024-08-252026-07-232025-05-123044.92closedmain
4StartCLI-00000004Ms.AvaSingh1962-03-02Fen7976BoulevardMapleemptyemptySaskatoonSKSaskatchewanS0A 4M7CAava8465@gmail.example+1-207-555-0195CTR-1000000004travel2024-09-042024-11-28expired590.50monthlydirect_debitAG-PR-0022024-08-062026-06-25emptyemptyemptymain
5StartCLI-00000005Ms.AriaSmith1987-04-08Fen6252StreetCedaremptyemptyAlmaQCQuebecH1N 9R6CAasmith@icloud.example+1-200-555-0126CTR-1000000005home2022-02-242027-08-30active1108.61monthlycredit_cardAG-PR-0042022-01-302026-03-13emptyemptyemptymain
6StartCLI-00000006Mr.EliasGreen1948-08-22Men7292PlaceBirchNorthemptyTerrebonneQCQuebecH0M 8P9CAeli933@yahoo.example+1-200-555-0148CTR-1000000006travel2024-12-022025-02-19expired1192.77monthlydirect_debitAG-PR-0012024-11-122025-04-23emptyemptyemptymain
7StartCLI-00000007Ms.PenelopeYoung1980-07-01Fen7479CourtOakemptyemptyDollard-des-OrmeauxQCQuebecH0Z 2J2CApenelopeyoung@hotmail.example+1-200-555-0133CTR-1000000007home2025-01-072026-11-21active2926.19monthlydirect_debitAG-PR-0032024-12-292025-01-24emptyemptyemptymain
8StartCLI-00000008Ms.EmilyPelletier1997-03-26Fen6530LaneLakeviewemptyemptyMontréalQCQuebecH0A 1X9CAemily.pelletier@gmail.example+1-200-555-0105CTR-1000000008travel2024-01-062024-02-26expired297.68monthlycredit_cardAG-PR-0012023-12-202025-03-30emptyemptyemptymain
9StartCLI-00000009Ms.GiannaSmith1992-01-13Fen8491BoulevardMountainviewempty388LongueuilQCQuebecH0G 4K7CAgianna.smith@icloud.example+1-200-555-0103CTR-1000000009auto2021-09-292026-12-10active1912.29monthlydirect_debitAG-PR-0012021-09-152026-06-092025-01-238026.55closedmain
10StartCLI-00000010Ms.GiannaWard1968-10-21Fen3086AvenueHighlandemptyemptyLévisQCQuebecH0J 0R3CAgia5002@bell.example+1-200-555-0144CTR-1000000010auto2023-06-202027-09-09active2585.27quarterlydirect_debitAG-PR-0022023-05-212026-06-20emptyemptyemptymain
3378SampledCLI-00003378Ms.LunaGreen1967-01-12Fen5907BoulevardSunsetemptyemptySaguenayQCQuebecH0K 3R9CAempty+1-200-555-0178CTR-1000003378home2025-08-172026-09-02expired2459.26annualcredit_cardAG-PR-0022025-07-182026-08-24emptyemptyemptymain
6473SampledCLI-00006473Ms.CharlotteTaylor1998-07-23Fen5722TerraceClearwateremptyemptyMississaugaONOntarioK0C 4L7CAtaylor.charlotte@gmail.example+1-201-555-0180CTR-1000006473auto2022-08-11emptyactive3061.06monthlydirect_debitAG-PR-0052022-07-172026-09-052026-07-27emptyopenmain
7809SampledCLI-00007809Ms.ElizabethLopez2006-09-30Fen8979WayHighlandSouth327ShawiniganQCQuebecH0Y 1S9CAelizabeth64@outlook.example+1-200-555-0199CTR-1000007809auto2026-09-10emptyactive3081.24monthlydirect_debitAG-PR-0012026-08-132026-09-08emptyemptyemptymain
16792SampledCLI-00016792Ms.EmilyThompson1966-08-24Fen4373StreetStationempty344GatineauQCQuebecH0E 3W6CAethompson.4@outlook.example+1-200-555-0198CTR-1000016792life2022-05-132027-08-12suspended4890.48monthlycredit_cardAG-PR-0032022-05-082026-08-30emptyemptyemptymain
35383SampledCLI-00035383Ms.AvaLi2006-08-05Fen5635RoadHillcrestNorthemptyMirabelQCQuebecH1K 3Z5CAaval.2@yahoo.example+1-200-555-0174CTR-1000035383auto2024-08-052026-09-04expired2209.18monthlydirect_debitAG-PR-0012024-08-052026-07-312026-07-282538.18closedmain
54302SampledCLI-00054302Ms.MadisonLopez1972-08-29Fen9DriveBrooksideemptyemptySherbrookeQCQuebecH0H 9W4CAlopez8279@yahoo.example+1-200-555-0108CTR-1000054302home2025-09-262026-09-08expired3100.12monthlydirect_debitAG-PR-0042025-09-262025-11-11emptyemptyemptymain
61892SampledCLI-00061892Ms.RileyCampbell1942-07-28Fen8388StreetSilverwoodemptyemptyJolietteQCQuebecH1J 4C9CAriley.campbell@icloud.example+1-200-555-0131CTR-1000061892home2019-07-172027-08-30active710.82monthlydirect_debitAG-PR-0012019-07-162026-07-09emptyemptyemptymain
62516SampledCLI-00062516Mr.WilliamWard1957-03-30Men9766CrescentHighlandEastemptyLongueuilQCQuebecH0G 0E9CAempty+1-200-555-0167CTR-1000062516home2025-05-142027-01-29active2771.50monthlydirect_debitAG-PR-0022025-04-292025-07-17emptyemptyemptymain
66809SampledCLI-00066809Ms.SophiaSmith1952-09-19Fen2194CircleMapleemptyemptyEdmontonABAlbertaT0B 2K9CAempty+1-203-555-0137CTR-1000066809travel2026-02-112026-03-12expired1143.64monthlycredit_cardAG-PR-0082026-02-112026-07-26emptyemptyemptymain
91990SampledQC-0001-0172Mr.LiamCollins09-08-1974Men4568DriveOakempty512GranbyQCQuebecH0V 7E9CAlcollins.5@outlook.example+1-200-555-0114CTR-3000010172auto30-07-202017-10-2026active992.46quarterlydirect_debitAG-QC-00515-07-202020-02-2023emptyemptyemptyquebec
99991EndQC-0001-8173Mr.MasonEvans16-03-1960Men4652AvenueHillcrestempty599DrummondvilleQCQuebecH0S 6C1CAmason_evans@outlook.example+1-200-555-0183CTR-3000018173home30-09-201707-05-2027active3410.59annualdirect_debitAG-QC-00527-09-201726-01-2023emptyemptyemptyquebec
99992EndQC-0001-8174Ms.OliviaMorris29-03-1940Fen5035WayCedaremptyemptySainte-JulieQCQuebecH1X 3V9CAoliviam.2@hotmail.example+1-200-555-0195CTR-3000018174life30-07-201910-11-2024expired7050.34monthlycredit_cardAG-QC-00108-07-201907-01-2022emptyemptyemptyquebec
99993EndQC-0001-8175Ms.AriaSmith26-05-1961Fen1631StreetBirchemptyemptySalaberry-de-ValleyfieldQCQuebecH1G 4T9CAaria.smith.8@gmail.example+1-200-555-0126CTR-3000018175auto28-01-202522-12-2026active2109.46monthlycredit_cardAG-QC-00231-12-202405-08-2026emptyemptyemptyquebec
99994EndQC-0001-8176Mr.DanielKing30-06-1965Men9150AvenueWillowNorthemptyLavalQCQuebecH0C 0G9CAdaniel.king.2@icloud.example+1-200-555-0131CTR-3000018176travel24-05-202512-08-2025expired234.26monthlydirect_debitAG-QC-00316-05-202523-06-2025emptyemptyemptyquebec
99995EndQC-0001-8177Ms.WillowThompson21-07-1965Fen1150StreetBirchemptyemptyLongueuilQCQuebecH0G 1G4CAwillow.thompson@gmail.example+1-200-555-0132CTR-3000018177home09-05-201809-06-2027active3445.06quarterlycredit_cardAG-QC-00406-05-201824-10-202006-04-201915049.99closedquebec
99996EndQC-0001-8178Ms.AveryAnderson23-11-1970Fen9256AvenueCedarNorth936GatineauQCQuebecH0E 3M5CAaveryanderson@gmail.example+1-200-555-0169CTR-3000018178life05-08-202326-05-2027active7059.96monthlydirect_debitAG-QC-00119-07-202322-04-2024emptyemptyemptyquebec
99997EndQC-0001-8179Ms.SophiaCollins12-10-1992Fen9476PlaceSouthridgeEastemptyRouyn-NorandaQCQuebecH1E 4S6CAscollins.9@gmail.example+1-200-555-0191CTR-3000018179auto08-05-2020emptyactive1902.48monthlycredit_cardAG-QC-00425-04-202003-06-2021emptyemptyemptyquebec
99998EndQC-0001-8180Mr.LucasWhite08-12-1994Men6596LaneSunsetempty994MontréalQCQuebecH0A 1H3CAlucas38@outlook.example+1-200-555-0131CTR-3000018180life19-09-201906-03-2024expired1039.38annualdirect_debitAG-QC-00103-09-201927-06-2022emptyemptyemptyquebec
99999EndQC-0001-8181Ms.NovaSingh05-03-2001Fen499StreetGreenfieldemptyemptyGatineauQCQuebecH0E 2W7CAnova-singh@yahoo.example+1-200-555-0115CTR-3000018181life05-05-202201-05-2027active1364.34monthlydirect_debitAG-QC-00323-04-202209-09-2023emptyemptyemptyquebec
100000EndQC-0001-8182Mr.LucasSmith26-12-1970Men9316DriveRosewoodempty416SaguenayQCQuebecH0K 7P1CAlucas72.2@yahoo.example+1-200-555-0152CTR-3000018182life15-09-201718-05-2027active852.38monthlydirect_debitAG-QC-00602-09-201724-01-2025emptyemptyemptyquebec

17 / Method

Analysis settings

Rules used for this analysis, so the result can be reproduced.

Scope
All 100,000 records
Missing values
absent, null, empty, blank, marker; markers: "N/A" "NULL"
Marker comparison
Whitespace trimmed; case-sensitive
Value normalization
Unicode composition (NFC): enabled; trim: enabled; collapse internal whitespace: enabled; comparison also with case folding: enabled, accents removed: enabled. Raw preview values are preserved.
Duplicate comparison
Exact raw values across every column; up to 2,000,000 distinct records compared
Type inference
At least 95.0% agreement across present values; numbers with dot or comma decimals, configured dates and true/false. Leading-zero identifiers remain text.
Date detection
Orders: YMD, MDY, DMY; separators: "-" "/" "." ; month names: en, fr; ambiguous order: unresolved, column evidence shown but never applied
Column types
The technical type, a detector matching at least 95.0% of present values, the length of the texts, the name of the column and, when the second pass counted them, the exact distinct values
Adaptive detection
Every detector on the first 10,000 distinct values of each column; detectors that recognized none of them, or at most 0.1% of them and none of the last 5,000, then skip the others, except about 1 value in 100. Number and date detection always test every value
Enumerations
At most 49 distinct values among at least 500 present values
Sensitive values
Shown in examples and the data sample
String lengths
Very short through 5; short through 20; medium through 50; long through 255; otherwise very long. Up to 10 examples per length are retained when the maximum is 50 or less.
Value representation
Complete through 50 distinct values; otherwise a reproducible sample of up to 100 distinct values, the ones with the smallest hash priorities
Row numbering
Data records start at 1, excluding the header. Quoted multiline values count as one record.
Row sample
The scan kept the first 50 records, up to 900 records drawn from the interior by a seeded hash, and the last 50; this report shows 10 at the start, 10 sampled and 10 at the end; raw values preserved
Patterns
Inferred from the sampled distinct values: up to 128 per column, values up to 64 characters, at least 2 distinct values per pattern
Observations
Rules version 2; at most 1,000 findings and 1,000 limits listed
Identifier uniqueness
Rule version 1, 2 columns. Exact duplicate rows read from the Scan's comparison of whole records. Copies of the rows without an ID counted by the second pass. The qualification reads no data and does not run Dedup.
JSON format
0.2.0 - revision 6 (experimental)
SOURCE SHA-25654b95b59720c792f4b1d32cf56073c44f0a6849d44536109b766468b60732327