Skip to content

Document policy for foreign expressions and code-switching #1001

Description

@nschneid

We have some documentation of the Foreign feature, a mention of foreign words in the X tag, and foreign expressions as an example of flat. But I can't find an overarching discussion of how to deal with foreign expressions.

Would the morphology overview be a good place for this?

Here is a crack at some text, including clarifications that were decided by the core group:

Foreign expressions and code-switching

A text that is primarily in one language may contain material that originated in another language.
UD offers a few options for annotating such material, which we term cross-lingual content.

Cross-lingual and cross-orthography metadata—translations, glosses, transliterations—may also be provided.

When it comes to the morphosyntactic annotation of an expression originating in another language, there are a few options:

Option 1: Code-switched analysis

A treebank may opt to fully analyze the cross-lingual content as if it were in a treebank for the source language.
This simulates a speaker with knowledge of the morphosyntax of both of the intermixed languages.
The language of any content analyzed in this manner should be specified on individual tokens
with the MISC feature Lang=CODE, as described here: this makes it clear which annotation guidelines
are being followed for the cross-lingual content so that the annotations can be properly validated.
The Foreign feature does not apply here.

  • That would be a/DET coup/NOUN d'/ADP état/NOUN .

Treebanks have wide latitude to decide what counts as a different language/code and whether to analyze its
structure or not. However, this strategy is generally not recommended for names mentioned in isolation.

Option 2: Borrowed analysis

Another option is to analyze the cross-lingual content as if it is part of the vocabulary
of the main language of the text. Tokenization principles of the main language,
not the donor language, would be expected to apply. Borrowed words are not marked
with Foreign=Yes because they are taken to be incorporated into the target language.
However, the donor language may be made explicit with the OrigLang feature in MISC.

For multiword expressions, the UPOS and morphological features of the expression as a whole
are copied to all the individual words, which are connected to the first word in a flat structure.
(For names, the subtyped relation flat:name may optionally be used.)

Nominals—including concept terms, personal names, and book titles—are frequently borrowed
and would typically be analyzed in this way. Other vocabulary may be considered borrowed as well.

  • Yeah , I think that would be kosher/ADJ .
  • That would be a/DET coup/NOUN d'état/NOUN .
  • We saw it on Al/PROPN Jazeera/PROPN .

Option 3: Foreign analysis

The third option is to treat the cross-lingual content as wholly unanalyzable foreign material.
Words should receive the feature Foreign=Yes in FEATS and be tagged as X.
Sequences of multiple foreign words are joined together by flat (optionally subtyped as flat:foreign).
In contrast to Option 2, this is best suited to phrasal idioms, quoted utterances, and metalinguistic mentions.
The foreign language, if known, is best made explicit with the OrigLang feature in MISC.

  • Well , c'est/X la/X vie/X .
  • " Lehitraot/X " means " see you later " in Hebrew .
  • " Monsieur/X , " she chided me , " nous/X ne/X parlons/X pas/X anglais/X dans/X cette/X classe/X !/PUNCT "

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions