Skip to content

Make Ibid handling compliant with the CSL spec#301

Merged
PgBiel merged 8 commits intotypst:mainfrom
wkordalski:ibid-test-interpretation
Apr 25, 2025
Merged

Make Ibid handling compliant with the CSL spec#301
PgBiel merged 8 commits intotypst:mainfrom
wkordalski:ibid-test-interpretation

Conversation

@wkordalski
Copy link
Copy Markdown
Contributor

@wkordalski wkordalski commented Apr 13, 2025

According to the CSL spec, especially:

Whenever position=”ibid-with-locator” tests true, position=”ibid” also tests true.

Confirmed by the discussion and by consistency between different styles that use Ibid
(they first check position="ibid-with-locator" and then position="ibid"),

Hayagriva handles Ibid position tests wrongly.
This PR fixes Ibid position tests handling.

See also: #299 (comment)

@PgBiel
Copy link
Copy Markdown
Contributor

PgBiel commented Apr 13, 2025

Thanks, can you add a local citeproc test demoing that this is working correctly? There are a few existing tests you can base yourself off of (let me know if you need help).

@wkordalski
Copy link
Copy Markdown
Contributor Author

Can I create two citations to the same bibliography entry with equal/different locator/supplement in the citeproc tests?
citeproc-js can do it with >>===== CITATIONS =====>> section, which is unsupported by Hayagriva tests.


Below there are test files I've ended with.
Hayagriva treats equal items in >>===== INPUT =====>> section as the same entry and different items as completely different entries, which is not what I need.

position_IbidIsNotIbidWithLocator.txt:

>>===== MODE =====>>
citation
<<===== MODE =====<<

>>===== RESULT =====>>
A 33; ibid.; ibid. 34
<<===== RESULT =====<<


>>===== CSL =====>>
<style
      xmlns="http://purl.org/net/xbiblio/csl"
      class="note"
      version="1.0">
  <info>
    <id />
    <title />
    <updated>2009-08-10T04:49:00+09:00</updated>
  </info>
  <citation>
    <layout delimiter="; ">
      <choose>
        <if position="ibid-with-locator">
          <group delimiter=" ">
            <text term="ibid" />
            <text variable="locator"/>
          </group>
        </if>
        <else-if position="ibid">
          <text term="ibid" />
        </else-if>
        <else>
          <group delimiter=" ">
            <text variable="title" />
            <text variable="locator"/>
          </group>
        </else>
      </choose>
    </layout>
  </citation>
</style>
<<===== CSL =====<<


>>===== INPUT =====>>
[
    {
        "id": "ITEM-1",
        "title": "A",
        "type": "book",
        "locator": "33"
    },
    {
        "id": "ITEM-1",
        "title": "A",
        "type": "book",
        "locator": "33"
    },
    {
        "id": "ITEM-1",
        "title": "A",
        "type": "book",
        "locator": "34"
    }
]
<<===== INPUT =====<<


>>===== VERSION =====>>
1.0.2
<<===== VERSION =====<<

and
position_IbidWithLocatorIsIbid.txt:

>>===== MODE =====>>
citation
<<===== MODE =====<<

>>===== RESULT =====>>
A 33; ibid.; ibid.
<<===== RESULT =====<<


>>===== CSL =====>>
<style
      xmlns="http://purl.org/net/xbiblio/csl"
      class="note"
      version="1.0">
  <info>
    <id />
    <title />
    <updated>2009-08-10T04:49:00+09:00</updated>
  </info>
  <citation>
    <layout delimiter="; ">
      <choose>
        <if position="ibid">
          <text term="ibid" />
        </if>
        <else-if position="ibid-with-locator">
          <group delimiter=" ">
            <text term="ibid" />
            <text variable="locator"/>
          </group>
        </else-if>
        <else>
          <group delimiter=" ">
            <text variable="title" />
            <text variable="locator"/>
          </group>
        </else>
      </choose>
    </layout>
  </citation>
</style>
<<===== CSL =====<<


>>===== INPUT =====>>
[
    {
        "id": "ITEM-1",
        "title": "A",
        "type": "book",
        "locator": "33"
    },
    {
        "id": "ITEM-1",
        "title": "A",
        "type": "book",
        "locator": "33"
    },
    {
        "id": "ITEM-1",
        "title": "A",
        "type": "book",
        "locator": "34"
    }
]
<<===== INPUT =====<<


>>===== VERSION =====>>
1.0.2
<<===== VERSION =====<<

@wkordalski
Copy link
Copy Markdown
Contributor Author

@PgBiel

@PgBiel
Copy link
Copy Markdown
Contributor

PgBiel commented Apr 16, 2025

Don't worry, I will get back to you soon. We are currently busy with our own work on Typst so we have less time to assign to reviews, but you can expect a response in the next few days.

@PgBiel
Copy link
Copy Markdown
Contributor

PgBiel commented Apr 23, 2025

Can I create two citations to the same bibliography entry with equal/different locator/supplement in the citeproc tests?
citeproc-js can do it with >>===== CITATIONS =====>> section, which is unsupported by Hayagriva tests.

Ideally we'd support it as well in citeproc tests, but you don't have to do this job if you don't want to. Instead, we can use a regular test like this one:

fn issue_48() {

@wkordalski
Copy link
Copy Markdown
Contributor Author

I also think that adding support for citations in citeproc tests would be ideal, but it is also a great amount of work, for which currently I don't have time. On the other hand, I think the ibid bug-fixes should not wait.

Thanks for pointing the "regular tests" way.

@wkordalski wkordalski force-pushed the ibid-test-interpretation branch from b3b8882 to e64551c Compare April 24, 2025 13:03
@wkordalski
Copy link
Copy Markdown
Contributor Author

I've also added a test that uses a non-synthetic CSL style. Maybe it is a good way to convince that the bug-fix changes Hayagriva in the right direction?

@PgBiel
Copy link
Copy Markdown
Contributor

PgBiel commented Apr 24, 2025

Fwiw there already is an official citeproc test for this exact case, so in theory Hayagriva already "has" this test: https://github.com/citation-style-language/test-suite/blob/b01762cc03bb53238bef479665fc0d169404449e/processor-tests/humans/position_IfIbidWithLocatorIsTrueThenIbidIsTrue.txt

Unfortunately, the test doesn't run as hayagriva doesn't support the position attribute in each citation item, which determines whether the item is an ibid or something else, apparently. However, I removed the position attribute anyway:

Details

>>===== MODE =====>>
citation
<<===== MODE =====<<




>>===== RESULT =====>>
Appleton “Tom Swift and His Electric Runabout” (1910).
Ibid.
<<===== RESULT =====<<


>>===== CITATION-ITEMS =====>>
[
    [
        {
            "id": "ITEM-1"
        }
    ],
    [
        {
            "id": "ITEM-1",
            "label": "page",
            "locator": "23"
        }
    ]
]
<<===== CITATION-ITEMS =====<<


>>===== CSL =====>>
<style
      xmlns="http://purl.org/net/xbiblio/csl"
      class="note"
      version="1.0">
  <info>
    <id />
    <title />
    <updated>2009-08-10T04:49:00+09:00</updated>
  </info>
  <citation>
    <layout suffix=".">
      <choose>
        <if position="first">
          <group delimiter=" ">
            <names variable="author">
              <name form="short" />
            </names>
            <text quotes="true" variable="title" />
            <date prefix="(" suffix=")" variable="issued">
              <date-part name="year" />
            </date>
          </group>
        </if>
        <else-if position="ibid">
          <text term="ibid" />
        </else-if>
        <else-if position="ibid-with-locator">
          <text term="ibid" />
          <label form="short" prefix=" " strip-periods="false" variable="locator" />
          <text prefix=" " variable="locator" />
        </else-if>
        <else>
          <text value="WTF?"/>
        </else>
      </choose>
    </layout>
  </citation>
</style>
<<===== CSL =====<<


>>===== INPUT =====>>
[
    {
        "author": [
            {
                "family": "Appleton",
                "given": "Victor"
            }
        ],
        "id": "ITEM-1",
        "issued": {
            "date-parts": [
                [
                    "1910"
                ]
            ]
        },
        "title": "Tom Swift and His Electric Runabout",
        "type": "book"
    }
]
<<===== INPUT =====<<


>>===== VERSION =====>>
1.0
<<===== VERSION =====<<

which showed that:

  1. The current output (without this PR) indeed seems to be broken;
  2. With this PR, we get the correct output, however the test still fails as hayagriva will apparently insert ibid. (lowercase) instead of Ibid.. I wonder if this is a citeproc-ism or we're missing something? Worth some thought.

@PgBiel
Copy link
Copy Markdown
Contributor

PgBiel commented Apr 25, 2025

In any case, I think we can then try to keep your originally proposed citeproc tests, however adapted to use citation-items, which seems to be the way to go for hayagriva to recognize the locator.

I believe that can't be done for the "real" style test, however. But to be honest, it is a bit fragile as it can start failing if the style updates. But I think we can just keep it as is for now (maybe with a bit more indicative of a name, e.g. ibid_in_deutsche_sprache_csl) and do something only if it really does start failing.

@wkordalski
Copy link
Copy Markdown
Contributor Author

wkordalski commented Apr 25, 2025

Done.


I'm confused with the position attribute in the citation item specification. Is it "a form of triggering ibid" alternative to specifying a locator? Or it overrides the result calculated from locator? What is the meaning of value of this attribute? Should I look here for the answer or is there a better source of truth? It feels that the position attribute got into the test suite by accident (and was copy-pasted into multiple locations) or was used in the past but not now.


I wonder if this is a citeproc-ism or we're missing something?

Possibly it is specified here: https://docs.citationstyles.org/en/stable/specification.html#citation

A note to CSL processor developers In note styles, a citation is often a sentence by itself. Therefore, the first character of a citation should preferably be uppercased when there is no preceding text in the note. In all other cases (e.g. when a citation is inserted into the middle of a pre-existing footnote), the citation should be printed as is.

Definitely a topic for a separate issue.


We can surround the citation with parentheses. Then "ibid." should not be uppercased (because parenthesis is the first character that should be uppercased, and uppercasing parentheses is a no-op). This way our test will match the official specification. What do you think?

Copy link
Copy Markdown
Contributor

@PgBiel PgBiel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is fine as is. Thank you!

@PgBiel PgBiel merged commit 73254c3 into typst:main Apr 25, 2025
2 checks passed
YDX-2147483647 added a commit to YDX-2147483647/clreq-gap-typst that referenced this pull request Oct 18, 2025
YDX-2147483647 added a commit to YDX-2147483647/clreq-gap-typst that referenced this pull request Oct 18, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants