The Gigatrees GEDCOM Validator (VGedX)

March 13, 2022 Applications Tim Forsythe

The Gigatrees GEDCOM Validator (VGedX) is a standalone Windows-x64 application similar to Gigatrees in configuration and operation, but is limited to the validation of GEDCOM files only. It can be downloaded here.

VGedX supports the following GEDCOM versions:

GEDCOM 5.5 Rev. 1 (January 2, 1996) (PDF)
GEDCOM 5.5 Rev. 2 (January 10, 1996)
GEDCOM 5.5.1 (PDF)
GEDCOM 5.6 (Draft) (PDF)
GEDCOM 7.0 (PDF)

Description

In addition to parsing GEDCOM files, GEDCOM validation requires that the GEDCOM file is validated against the specification requirements of both the GEDCOM grammar (line and file syntax) and the GEDCOM dictionary (record format, data types, data formats and data values). In this respect, VGedX is only a partial validator in that it does not in all cases, validate data formats or data values. It also does not have complete coverage of the validation tests listed in the Validation Details below. This is because in many cases, the data types and values defined in the specification are incomplete or inconsistent. VGedX will validate each record's defined properties, i.e. minimum and maximum occurrences, id reference types, etc. It will not validate any enumerated values. Data consistency testing is not part of a GEDCOM validation.

Command Line Options

VGedX supports several command line options. The input file and output path are optional here and can be set via the <Main> configuration option show below. The log is always optional.

Config File

-c config.xml

Loads an individual configuration file.

Input File

-i family.ged

Loads a GEDCOM file.

Output Path

-o web

Sets the output path.

Log

-l build.log

Generates a log file.

The distribution file contains all five GEDCOM test files ( gedcoms/VGedXGedcom55r1TestFile1_0.ged, gedcoms/VGedXGedcom55r2TestFile1_0.ged, gedcoms/VGedXGedcom551TestFile1_0.ged, gedcoms/VGedXGedcom56TestFile1_0.ged and gedcoms/VGedXGedcom70TestFile1_0.ged ). These test files can be used to test VGedX's importing capabilities. The test files use typical values only, and are therefore not useful for testing boundary uses cases.

Configuration

To configure VGedX you must provide the path to your GEDCOM file and the output folder either using the command line options listed above, or here. File and folder paths may be entered as absolute or relative paths. Relative paths are relative to the folder where the application file ( vgedx.exe ) resides, called the working directory.

<Main> Options

[ ]

Expects an absolute or relative file path.

[ ]

Expects an absolute or relative file path.

VGedX categorizes its validation results as Errors, Warnings and Alerts. Errors cannot be hidden.

<Validation> Options

[ true ]

Shows validation warnings. It is sometimes useful to turn off warnings temporarily to reduce the size of the table used to hold the validation results.

[ true ]

Shows validation Alerts.

[ false ]

Shows records that are not referenced elsewhere in your GEDCOM file, making it effectively unused.

[ true ]

Shows records that start with an underscore ( i.e. _PRIV ). These are not used internally by the GEDCOM standards and therefore come from various vendors. As such they may not be recognized by an importing application.

[ true ]

Trailing delimeters such as spaces or tabs, are a violation of the GEDCOM standard, but cause no harm and were prevalent in my export testing of many genealogy applications.

[ true ]

Often when a particular tag causes a warning, thousands of that same tag will as well. If you are only interested in looking at the first occurance, you can disable this, reducing the validation table size significantly.

[ true ]

VGedX builds tag lists for each record ( i.e. INDI@.BIRT.@SOUR ), where and ampersand appearing after a tag indicates a level 0 record containing a record id ( INDI, SOUR, OBJE, FAM, etc.), and an ampersand appearing before the tag indicates it is a reference to another record and contains an xref_id. When disabled, these are not displayed, making it tricky to understand where the issue occured in the file without looking up the line number in the file. On the otherhad, crazy long tag lists can cause the validation table to display less ... efficiently.

[ 50 ]

When text is display in the validation table, it is limited in length and if cropped an ellipsis will be appended to the text. You can control the width of this text here.

Example

In the following example, the GEDCOM file and the output path are both located in the working directory.

<Options>

  <Main>
    <GedcomFile> gedcoms/VGedXGedcom70TestFile1_0.ged </GedcomFile>
    <OutputPath> vgedx                                </OutputPath>
  </Main>
  
  <Validation>
    <ShowValidationWarnings>      true  </ShowValidationWarnings>
    <ShowValidationAlerts>        true  </ShowValidationAlerts>
    <ShowUnusedRecords>          false  </ShowUnusedRecords>
    <ShowUserDefinedRecords>      true  </ShowUserDefinedRecords>
    <ShowTrailingDelimiters>      true  </ShowTrailingDelimiters>
    <ShowTagWarningDuplicates>    true  </ShowTagWarningDuplicates>
    <ShowTagLists>                true  </ShowTagLists>
    <ValidationDataWidth>           50  </ValidationDataWidth>
  </Validation>

</Options>

Sample

The following image shows a portion of the validation report. In the GEDCOM file section are listed details about the file itself. It is important to make sure that the File Line Count matches the number of lines in the actual file. In rare cases, when creating continuation tags that require splitting text strings (probably wide characters), some exporting applications create non-printable characters that appear to VGedX as an end-of-file marker, preventing the file from being completely parsed. A GEDCOM revision will be displayed only if detected. In the GEDCOM status section, status types are color coded, and the table can be sorted by clicking on any of the table headings.

Validation Details

Data Formats

Symbols

() parentheses  = grouped components
[] brackets     = optional components
*  astricks     = multiple occurrences of a component
-  dash         = range of values of a component
|  pipe         = component or

Characters

Character               ASCII value
=========               ===========
tab                     = 0x09
line feed               = 0x0A
carriage return         = 0x0D
space                   = 0x20
exclamation point (!)   = 0x21
cross hatch (#)         = 0x23
colon (:)               = 0x3A
ampersand (@)           = 0x40
underscore (_)          = 0x5F

Character Sets

Character set                  ASCII range
=============                  ===========
non-zero number digit (1-9)  = (0x31 - 0x39)
number digit (0-9)           = (0x30 | non-zero number digit )
uppercase char               = (0x41 - 0x5A)
alpha char (a-zA-Z_)         = uppercase char | (0x61 - 0x7A) | 0x5F
non-alpha char               = (0x21 - 0x2F) | (0x3A - 0x3F) | (0x5B - 0x5E) | (0x7B - 0x7E) | (0x80 - 0xFE) | 0x60
not-at char                  = (%x09 | %x20-3F | %x41-10FFFF)
non-eol char                 = (%x09 | %x20-FFFF | %x10000-10FFFF) - [%x10000-10FFFF not tested]

Character Groups

alphanum                = (alpha char | number digit)	
printable character     = alphanum | non-alpha char | space | cross hatch

Strings

double-at string (@@)     = ampersand + ampersand
number string             = number digit + [number digit]*
alphanum string (G5 only) = alphanum + [alphanum]*
alphanum string (G7 only) = (uppercase char | number digit | underscore)
mixed string (G7 only)    = uppercase char [+ *(alphanum string)] | underscore + *(alphanum string)
pointer id (G5 only)      = (alphanum | exclamation point) + [printable character]*
pointer string (G5 only)  = ampersand + pointer id + ampersand 
pointer string (G7 only)  = ampersand + (mixed string | VOID) + ampersand
embedded id string        = ampersand + [pointer id +] exclamation point + pointer_id + ampersand 
escape string             = ampersand + cross hatch + (printable character | double-at string)* + ampersand + [space] + (printable character)* 
value string              = printable character + [printable character]*
data string (G5 only)     = (value string | escape string) [+ (value string | escape string)]*
data string (G7 only)     = (not-at char | double-at string) + [non-EOL char]*

delimiter                 = space
terminator                = carriage return | line feed | (carriage return + line feed) | (line feed + carriage return)
whitespace                = ([tab]* + [space]* + [terminator]*)*

Validation Tests

GEDCOM Validation testing includes two types of tests, GEDCOM data format and GEDCOM form.

GEDCOM 5 Line Syntax

All of the supported GEDCOM Dictionaries use the same data format, which defines a line as having the following syntax:

line = [whitespace +] level + [delim + record_id +] delim + tag + [delim + (reference_id | line_value) +] terminator

GEDCOM Grammar Tests

The following is a list of requirements of the GEDCOM 5 ( G5 ) and GEDCOM 7 ( G7 ) grammars. Unsupported tests will be noted. String lengths are measured in characters, not bytes.

The file must use UTF-8 file encoding (G7 only)
End-of-line markers must be a carriage return, a line feed, or a carriage return fallowed by a life feed.
End-of-file markers must be consistent throughout the file.
Blank lines are prohibited (G7 only)
Leading spaces are prohibited (G7 only)

The level is a number string.
Level numbers should not contain leading zeroes.
The minimum level number is 0.
The maximum level number is 99. (G5 only)
The maximum level number increment is 1.
The level must be followed by a delimiter.

A record_id can be a pointer id string (or an embedded id string - G5 only).
The length of a record_id is between 3 and 22 characters (G5 only)
The minimum length of a record_id is 1 character. (G7 only)
The record_id must be followed by a delimiter.
The record_id must be unique to the file.
A record_id only required when a matching reference_id exists.
The record_id must not be @VOID@.

1 @!O1@ OBJE (I1 is implied)

1 @I1!O1@ OBJE (duplicates not allowed)

The tag is a alphanum string. (G5 only)
The tag is a mixed string. (G7 only)
The length of the tag is between 1 and 31 characters. (G5 only)
The first 15 characters of the tag must be unique. (G5 only)
The tag must be preceded by a delimiter.

A reference_id is a pointer string.
The length of a reference_id is between 3 and 22 characters. (G5 only)
The reference_id must be preceded by a delimiter.
The reference_id must be followed by a terminator.
A record_id must exist in the file of a matching reference_id is found, unless a colon is present. (G5 only).
A record_id must exist in the file of a matching reference_id is found, unless the reference_id is @VOID@ (G7 only).
If the reference_id contains an exclamation point, the record_id must exist in an embedded record contained within the same logical record. (G5 only)

for example (G5 only):

0 @I1@ INDI
1 @I1!O1@ OBJE
1 OBJE @I1!O1@
1 OBJE @!O1@ (I1 is implied)

0 @I2@ INDI
1 OBJE @I1!O1@ (not allowed)

A line_value is a data string.
The line_value must be preceded by a delimiter.
The line_value must be followed by a terminator.
The line_value must not have trailing spaces. (G5 only)
If an ampersand is desired as part of the line_value, it must be included as a double-at string (i.e. name@@school.edu). (G5 only)
If an ampersand is the first character of a line_value, it must be included as a double-at string.

The maximum length of a line is 255 characters. (G5 only)
The maximum length of a logical record is 32 kilobytes (logical records are delineated by level numbers equal to 0 (zero)). (G5 only)

GEDCOM Dictionary Tests

To validate the GEDCOM dictionary, VGedX compares the structure of the logical records to the GEDCOM dictionary template associated with its GEDCOM version. It also validates general GEDCOM dictionary constructs common to all supported GEDCOM versions.

The GEDCOM version must be either "5.5", "5.5.1", "5.6" or "7.0".
Each line should match the GEDCOM dictionary template unless the line has a user defined tag beginning with an underscore.
Each record_id should be referenced from within the same file.
If the template expects a record_id, then the line must have a record_id of the same type.
If the template expects no record_id, then the line must not have a record_id.
If the template expects a reference_id, then the line must have a reference_id of the same type.
If the template expects no reference_id, then the line must not have a reference_id.
If the template defines a minimum number of record occurrences, then the record should not have fewer.
If the template defines a maximum number of record occurrences, then the record should not have more.
If the template defines a minimum line_value length, then the line_value should not be shorter.
If the template defines a maximum line_value length, then the line_value should not be longer.

Tim Forsythe is a retired software engineer. He is married and living in Sonoma County, California.

#GEDCOM, #VGedX