Improving citations

[ennoborg]

On RootsTech 2022, I saw a presentation by Dave Thaler about a possible future for citations, in GEDCOM 7, with support for EE, and hierarchies, and derivatives, which looked very interesting, because I'm sort of done with the current standard.

Is there a way to continue that here? Would it fit in a version 7.0.X, or 7.x?

[Norwegian-Sardines]

Like so many things that were not fixed in v7.0.x, sources and source citations need to be expanded to include documentation about tag usage for various general source types. However, saying support for EE is a not telling me much! In my opinion, EE is a very over the top with its 100s of variations and it needs to be distilled down to be useful for people without a Library Science background. To make the guidance of EE valuable and usable in a database, data field rules need to be created, and a general set of use cases need to be developed and documented in the GEDCOM Standard before an application can interface with the database to populate specific data fields.

[clarkegj]

Ennoborg and Norwegian-Sardines,

We appreciation you comments and contributions to Github.com/Familysearch/GEDCOM. Please check out the Project Teams for future versions of FamilySearch GEDCOM that we are organizing. (see https://gedcom.io/community. I recommend that you apply to participate in the Sources Team by sending an email to [email protected] and put "Sources" in the subject.

[ennoborg]

I agree with your comment on EE. It's so much over the top, that we gave up on implementing John Yates' public domain derivative in Gramps, because even in that subset there were too many different fields, especially when a program needs to be translated in 40+ languages.

And from where I work, in The Netherlands, I see that the way that archive catalogs work, in my country, and in England, France, Germany, and the USA, to name the countries where I have relatives, can easily be translated to a reasonable number of fields, provided that we accept the hierarchies (and derivatives) as they are already defined in the SourceDescription and SourceReference tructures like they exist in GEDCOM X.

I'd be happy to contribute by writing use cases for the countries where my ancestors lived.

[Norwegian-Sardines]

IMHO. I think that the way the few programs I have seen using EE (Evidence Explained: Elizabeth Shown Mills) for citation generation and its subsequent storage in a GEDCOM file leaves a lot to desire. These programs take liberties on creating "user defined tag" all over the place (in many cases ignoring the actual construct that they should use) but can't create a set of user defined tags that would allow for transfer of citation data to other programs that use a different citation scheme!

While I'm not a Librarian, I did study Library Science and Cataloging in particular as part of my Masters Degree. EE is not the only way to implement source citations, EE is well received and has a lot of merits, while capturing all of the nuances of citation writing, but as I said it has a lot of detail that 99% of Family Historians and Arm Chair genealogists just don't care about.

On the other hand the current Source and Source_Citation scheme provided in GEDCOM is rooted in the days when sources were books and manuscripts. For the most part this scheme works with little problems or missing data points but some things do get lost with "born digital" data.

In the case of the stored citation data in GEDCOM internationalization should only occur on the labels for the data, not the data itself. Labels should not be stored in the GEDCOM itself, but each tag should contain a specific type of data, in the EE quick reference their are only about 18 to 20 data points that need to be captured, many of which are already part of the SOURCE_RECORD and SOURCE_CITATION structure. Some of these data-points need new definitions (and probably new names) to fulfill the larger need of citations of "born digital" data.

My personal assessment is that GEDCOM should not implement EE specifically but should implement tags/fields that allow for the capture of all citation data and add a new tag that describes the source and citation that the captured data represents (read this as a SOUR:TYPE tag). This way, if a receiving programs has not "bought into" the EE scheme, but uses a different citation scheme it can extract the data and use it correctly for their scheme.

[ennoborg]

As far as I can remember, and I think that's from a discussion on the RootsDev group, Elizabeth Shown Mills never saw EE as a software design, in the way it's implemented in the programs that we know, and I assume that we're referring to the same programs, like Legacy, and RootsMagic, which are the ones that I know best.

But at the same time, I know that many people like to use EE as a guideline, and even FamilySearch mentions it. And in this case, I mentioned it, because @dthaler referred to it in this part of the presentation on RootsTech:

https://youtu.be/-yqlHAd3eNk?t=427

In this presentation, Dave also makes it clear that EE is not the only way, and I fully agree with that. In fact, when I want to create a publication for the Dutch genealogical society, I will have to comply with their standards, which are way more concise than the ones that I see in publications that use scientific standards like APA, Chicago, or MLA.

I must add that in most situations, I don't really care much about formatted citations, because I concentrate on the research itself, for which I like to record provenance and containment, which is quite hard with the quite rigid structures that we have in GEDCOM 5.5.1, also because there aren't enough fields to store what you call data points. Most sources that I see often need less than 10, but I agree that you need more variables, as I like to call them as a software developer, because of all the variations that we see 'in the wild'. And for me, EE is a nice inventory of what you can see, in the wild.

When you allow for source records to record containment and provenance, you can simplify things a bit by letting the SOURCE_RECORD have references to other source records, as a container, or a source, and allowing all tags that can appear in the SOURCE_CITATION in that too. That will most probably not be enough, but when you realize that many fields that you find in EE (and CSL) are actually used as a shortcut to reflect containment and provenance, I think that you may also find that when you allow that, a dozen fields will probably be enough for most.

And when we can keep the number of tags to a dozen, or 18 or 20, or a bit more, translation will be way more easy than with a verbatim implementation of a guideline that was never meant to be a software standard at all.

[clarkegj]

We appreciation you comments and contributions to Github.com/Familysearch/GEDCOM. Please check out the Project Teams for future versions of FamilySearch GEDCOM that we are organizing. (see https://gedcom.io/community. I recommend that you apply to participate in the Sources Team by sending an email to [email protected] and put "Sources" in the subject.

[tychonievich]

During the 7.0 pre-release design phase I was a part of a team discussing sources, citations, and hypothesis-oriented structures. While that group did not make an official recommendation, it was my personal impression that

1. Citations are more complicated than we were ready to "solve," and waiting for us to do so would have delayed 7.0's release unacceptably long.

2. Many other groups have looked at family history citations and either abandoned the project as too large to handle or created a specification that failed to meet many users' requirements.

3. Even if we produced a "good" citation model, few if any applications would be ready to adopt it. We probably need a stand-alone model proposal for application developers to try out before we try to incorporate it into a standard.

To the best of my knowledge, formal discussion of citations in FamilySearch GEDCOM have not resumed since the release of 7.0.

In addition to my role with the GEDCOM specification, I also chair FHISO. In the FHISO annual general meeting last September citations was identified as our top priority, and some progress has been made by FHISO on that since then. We have produced a survey of possible designs field and are actively exploring two approaches identified there: (a) assembling a semantic markup model from various third-party recommendations and (b) extending the CSL standard from an academic-focused property-set model to a genealogy-enabled graph-of-layers model. We also hand-crafted a few graph-of-items examples for discussion and liked the result, but are not pursuing that as actively because we are aware of less tooling to support implementation.

[ennoborg]

Thank you for responding, and I fully understand what you say. I've been involved in many discussions on Better GEDCOM, and RootsDev, and FHISO mailing list, so I know what you're talking about.

When I think about the past, standards have almost always been started by single entities, not committees, like the Compact Cassette, invited in my country, by Philips, or the CD, by a party of two, SONY and Philips, or a country, like France, that forced the whole world to use SCART. And that's just like FamilySearch has always been able to set the standard by creating GEDCOM, and distributing a killler app, PAF. There are numerous other examples, like Linux, and Windows, which were also set by a de-facto leader, not by a committee.

I read the survey, but I don't see any 3rd party references over there. Would you consider the model that FamilySearch already has for the citations that are now generated on-site? Or the model used by the Dutch OpenArch.nl, as it's supported by Bob Coret?

[tychonievich]

I'd be happy to review those, but so far as I can tell from the outside both of those are slightly-formatted text only. Can you point me to any back-end data structures or documentation about how those citations are stored, rather than how they are presented?

We did review BibTex, CSL/CiteProc (invented by OpenOffice), Zotero, Mendeley, and Quigga, all of which are list-of-items systems with no handling of layers. We also looked at Stemma (by Tony Proctor) and of course at GEDCOM and GEDCOM-X.

[ennoborg]

I can do that for OpenArch.nl, for which you can find some documentation on Bob's own site:

https://a2a.coret.org/

To my regret most of the other documentation has disappeared from the web, but I think that this should suffice. Note that the model documentation is Dutch, so you may just scan it to see some examples. The schema documentation is English however, and the example links below the URL field can also display some good examples.

A2A is a list-of-items system, just like the others.