word segmentation and lemmatization should ignore <orig> when it's alone

[emylonas]

when the <orig> element appears alone, and not as a child of <choice> the content should be ignored. Characters inside that type of <orig> are not words. They represent something that is not recognizable as a word.

Examples:
masa0836 has the string <orig>CB</orig> this appears in the word list
same in masa0838
caes0062 has the string <orig>C</orig> and <orig>DO</orig> seems to not be in the list

[atbradley]

Do we want these to have @xml:ids?

[emylonas]

yes, the xml:ids help with context for parsing (so orig not useful) and also for creating kwic views, where the orig will be useful.orig not useful) and also for creating kwic views, where the orig will be useful. If it's in the segmented div, it should have an xml:id. orig doesn't have an xml:lang, and num probably should, so that will have to be copied as for <w>

…

On Tue, Jul 6, 2021 at 9:07 AM Adam Bradley ***@***.***> wrote: Do we want these to have @xml:ids? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#119 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAOA7ZK34GD3BR5UFCNEWMDTWL52DANCNFSM4YIW76CA> .