OPENNLP-1782: Add tagging examples to verify French POS model by meriam2303 · Pull Request #863 · apache/opennlp

meriam2303 · 2025-09-29T17:21:30Z

French Model draft to test + Arabic & Maghrebi commented out

For documentation related changes:

Have you ensured that format looks appropriate for the output in which it is rendered?

mawiesne · 2025-09-30T06:59:03Z

Thx @meriam2303 for the PR. Let's see if it passes the tests.

mawiesne · 2025-09-30T07:01:14Z

@meriam2303 It seems there is a syntax error:

POSTaggerMEIT.java. 157:6: mismatched input 'Arguments' expecting ')'

Could you check, correct it and push a fix to the same branch?

Please also add a new static constant to that test:

private static final String FRENCH = "fr";

See other constants close to the class definition (POLISH, GERMAN, ENGLISH...)

pazairfog · 2025-10-06T10:29:13Z

Hi,

There are still some indentation errors for the French data.

diff --git a/opennlp-core/opennlp-runtime/src/test/java/opennlp/tools/postag/POSTaggerMEIT.java b/opennlp-core/opennlp-runtime/src/test/java/opennlp/tools/postag/POSTaggerMEIT.java
index 0963f67e..3bd10889 100644
--- a/opennlp-core/opennlp-runtime/src/test/java/opennlp/tools/postag/POSTaggerMEIT.java
+++ b/opennlp-core/opennlp-runtime/src/test/java/opennlp/tools/postag/POSTaggerMEIT.java
@@ -135,15 +135,15 @@ public class POSTaggerMEIT {
       Arguments.of(POLISH, 0,
         "Działacze stosowali też różne formy nacisku na polski konsulat , żeby zaopiekował się " +
         "bezrobotnymi z Polski albo dał im choćby na bezpłatny bilet do kraju .",
-          new String[]{"NOUN", "VERB", "PART", "ADJ", "NOUN", "NOUN", "ADP", "ADJ", "NOUN", "PUNCT", "SCONJ", 
-            "VERB", "PRON", "ADJ", "ADP", "PROPN", "CCONJ", "VERB", "PRON", "PART", "ADP", "ADJ", "NOUN", 
+          new String[]{"NOUN", "VERB", "PART", "ADJ", "NOUN", "NOUN", "ADP", "ADJ", "NOUN", "PUNCT", "SCONJ",
+            "VERB", "PRON", "ADJ", "ADP", "PROPN", "CCONJ", "VERB", "PRON", "PART", "ADP", "ADJ", "NOUN",
             "ADP", "NOUN", "PUNCT"}),
       // via: @kinow
       Arguments.of(CATALAN, 1,
         "Un gran embossament d'aire fred es comença a despenjar cap al centre d'Europa.",
           // OpenNLP, different at: idx pos 2, 3, 5, and 13(+14) -> however, only pos 5 is "wrong" (ref)
           new String[]{"DET", "ADJ", "NOUN", "ADP", "NOUN", "ADJ", "PRON", "VERB", "ADP", "VERB", "NOUN",
-              "ADP+DET", "NOUN", "ADP", "PROPN", "PUNCT"}),
+            "ADP+DET", "NOUN", "ADP", "PROPN", "PUNCT"}),
       // REFERENCE ("gold"):
       // "DET", "ADJ", "NOUN", "ADP", "NOUN", "ADJ", "PRON", "VERB", "ADP", "VERB", "NOUN", "ADP+DET",
         // "NOUN", "ADP", "PROPN", "PUNCT"})
@@ -153,22 +153,22 @@ public class POSTaggerMEIT {
         // "NOUN", "PROPN", "PROPN", "PUNCT"
         // ok! ,  ok! ,  ??? ,  ???   ,  ok!  ,  ok! ,  ok!  ,  ok!  ,  ok! ,  ok!  ,  ok!  ,  ok!  +  ok! ,
         // ok!  ,  ???   ,  ok!   ,  ok!
-      // via: @meriam2303 , original by Guillaume Musso:
-      // La jeune fille et la nuit, S.469 
+      // via: @meriam2303, original by Guillaume Musso:
+      // La jeune fille et la nuit, S.469
       Arguments.of(FRENCH, 0,
-      "Vivre avec elle me faisait souffrir, mais vivre sans elle m'aurait tué.",
-      new String[]{"VERB","ADP","PRON","PRON","AUX","VERB","PUNCT","CCONJ","VERB","ADP","PRON","PRON","AUX",
-          "VERB","PUNCT"})
+        "Vivre avec elle me faisait souffrir, mais vivre sans elle m'aurait tué.",
+          new String[]{"VERB", "ADP", "PRON", "PRON", "AUX", "VERB", "PUNCT", "CCONJ", "VERB", "ADP", "PRON",
+            "PRON", "AUX", "VERB", "PUNCT"})
       // via @meriam2303, original by Hind Choueykh Ben Salah
       // التجريد في الشّعر العربي , S. 42
       //Arguments.of(ARABIC,0,
       //"عشق أبو نواس جارية تدعى جنان",
-      //new String[]{"VERB","PROPN","NOUN","VERB","PROPN"})  
+      //new String[]{"VERB","PROPN","NOUN","VERB","PROPN"})
       // via @meriam2303, original by Mohamed Laarousi Elmetoui
       // التوت المر , S.7
       //Arguments.of(MARGHREBI_ARABIC_FRENCH,0,
       //"Wassa3 belek ya baba...",
-      //new String[]{"VERB","NOUN","ITNJ","NOUN","PUNCT"})    
+      //new String[]{"VERB","NOUN","ITNJ","NOUN","PUNCT"})
     );
   }
 }

rzo1

Fixed checkstyle but currently fails due to missing model (?)

Error:  Tests run: 8, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.277 s <<< FAILURE! -- in opennlp.tools.postag.POSTaggerMEIT
Error:  opennlp.tools.postag.POSTaggerMEIT.testPOSTagger(String, int, String, String[])[8] -- Time elapsed: 0.005 s <<< ERROR!
java.lang.NullPointerException: Cannot invoke "opennlp.tools.tokenize.Tokenizer.tokenize(String)" because the return value of "java.util.Map.get(Object)" is null
	at opennlp.tools.postag.POSTaggerMEIT.testPOSTagger(POSTaggerMEIT.java:66)

rzo1 · 2025-10-14T08:32:52Z

Fixed the test setup. Now we have

Vivre_VERB avec_ADP elle_PRON me_PRON faisait_VERB souffrir_VERB ,_PUNCT mais_CCONJ vivre_VERB sans_ADP elle_PRON m'_PRON aurait_AUX tué_VERB ._PUNCT 
Vivre[VERB] <-- OK
avec[ADP] <-- OK
elle[PRON] <-- OK
me[PRON] <-- OK
faisait[VERB] <-- NOK, pos=4
souffrir[VERB] <-- OK
,[PUNCT] <-- OK
mais[CCONJ] <-- OK
vivre[VERB] <-- OK
sans[ADP] <-- OK
elle[PRON] <-- OK
m'[PRON] <-- OK
aurait[AUX] <-- OK
tué[VERB] <-- OK
.[PUNCT] <-- OK

which should be AUX according to the provided reference.

Think it is an edge case here:

Actually, faisait is the imperfect of “faire”. Here it functions as a semi-auxiliary in faisait souffrir. Tagging it as AUX is acceptable in Universal Dependencies because “faire” + infinitive is considered an auxiliary construction. (not a native French speaker though).

Tried some other (online) taggers, which will label faisat as VERB.

meriam2303 · 2025-11-09T19:08:50Z

opennlp-core/opennlp-runtime/src/test/java/opennlp/tools/postag/POSTaggerMEIT.java

according to the dictionary of the french academy AUX is “a verb that functions as a grammatical tool used to build the compound tenses of other verbs.” and faire is considered a semi-auxiliare. Because this option is not available and because "faire" doesn't build a compound tense of the verb "souffrir" it is safer and more correct to consider faire a verb NOT an auxiliare.

Note: current CI failure is unrelated to this change (403, sourceforge)

mawiesne · 2025-11-11T13:51:27Z

thx @meriam2303 for your first time open-source contribution. All checks passed which is why I've merged the PR.

* adds French sample sentence and pos tags, incl. arabic+maghrebi stub examples for existing tests * adds French constant * inits French resources for test context --------- Co-authored-by: Richard Zowalla <rzo1@apache.org> (cherry picked from commit 237a771)

meriam2303 added 2 commits September 1, 2025 16:28

french+arabic+maghrebi

0b7a278

arabic+maghrebi_arabic_french rauskommentiert

56ccbbe

meriam2303 marked this pull request as draft September 29, 2025 17:25

mawiesne changed the title ~~french model~~ Add tagging examples to verify French POS model Sep 30, 2025

mawiesne added the tests Pull requests that add or update test code label Sep 30, 2025

mawiesne self-assigned this Sep 30, 2025

mawiesne requested a review from rzo1 September 30, 2025 07:03

added french constant and fixed the syntax error

b69b920

mawiesne marked this pull request as ready for review September 30, 2025 15:51

Fix checkstyle

6fe239a

rzo1 reviewed Oct 14, 2025

View reviewed changes

Init FRENCH resources for test context

72c7965

mawiesne changed the title ~~Add tagging examples to verify French POS model~~ OPENNLP-1782: Add tagging examples to verify French POS model Oct 15, 2025

changed faisait to verb

814ab2d

meriam2303 commented Nov 9, 2025

View reviewed changes

mawiesne approved these changes Nov 11, 2025

View reviewed changes

mawiesne merged commit 237a771 into apache:main Nov 11, 2025
13 of 39 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OPENNLP-1782: Add tagging examples to verify French POS model#863

OPENNLP-1782: Add tagging examples to verify French POS model#863
mawiesne merged 6 commits intoapache:mainfrom
meriam2303:patch-1

meriam2303 commented Sep 29, 2025 •

edited

Loading

Uh oh!

mawiesne commented Sep 30, 2025

Uh oh!

mawiesne commented Sep 30, 2025 •

edited

Loading

Uh oh!

pazairfog commented Oct 6, 2025

Uh oh!

rzo1 left a comment •

edited

Loading

Uh oh!

rzo1 commented Oct 14, 2025

Uh oh!

meriam2303 Nov 9, 2025

Uh oh!

rzo1 Nov 9, 2025

Uh oh!

Uh oh!

mawiesne commented Nov 11, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

meriam2303 commented Sep 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

For documentation related changes:

Uh oh!

mawiesne commented Sep 30, 2025

Uh oh!

mawiesne commented Sep 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pazairfog commented Oct 6, 2025

Uh oh!

rzo1 left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rzo1 commented Oct 14, 2025

Uh oh!

meriam2303 Nov 9, 2025

Choose a reason for hiding this comment

Uh oh!

rzo1 Nov 9, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mawiesne commented Nov 11, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

meriam2303 commented Sep 29, 2025 •

edited

Loading

mawiesne commented Sep 30, 2025 •

edited

Loading

rzo1 left a comment •

edited

Loading