Module talk:Footnotes

Consolidated updates

Renata3 there's a few updates waiting to be done, could you add the following entries?
['CITEREFMarshall1969'] = {'Lancashire & Yorkshire Railway 1'}, ['CITEREFMarshall1970'] = {'Lancashire & Yorkshire Railway 2'}, ['CITEREFRowling1997'] = {'HP1ref'}, ['CITEREFRowling1998'] = {'HP2ref'}, ['CITEREFRowling1999'] = {'HP3ref'}, ['CITEREFRowling2000'] = {'HP4ref'}, ['CITEREFRowling2003'] = {'HP5ref'}, ['CITEREFRowling2005'] = {'HP6ref'}, ['CITEREFRowling2007'] = {'HP7ref'}, ['CITEREFShiji'] = {'Cite Shiji'}, ['CITEREFSima'] = {'Cite Shiji'}, ['CITEREFSimaSima'] = {'Cite Shiji'},

Thanks, -- LCU ActivelyDisinterested «@» °∆t° 10:52, 5 August 2024 (UTC)

 Done Renata•3 03:14, 6 August 2024 (UTC)

Thanks Renata. -- LCU ActivelyDisinterested «@» °∆t° 10:23, 6 August 2024 (UTC)

Thank you -- Nempnet (talk) 13:57, 8 August 2024 (UTC)

Notice: discussions on WMDE sub-reference project in progress at Meta

The WikiMedia sub-referencing project (parent project: Reusing references) is having multiple discussions about the development of a sub-referencing feature by WikiMedia Deutschland Engineering. Your feedback would be welcome at any of the discussons at m:Talk:WMDE Technical Wishes/Sub-referencing. Mathglot (talk) 20:58, 23 August 2024 (UTC)

Another whitelist request

Hi @Renata3, there is more Tolkien (no surprise). I may or may not have gotten the format right this time:

['CITEREFTolkien2017']={'ME-ref|B&L'}

Thanks, Andy02124 (talk) 16:50, 26 August 2024 (UTC)

Probably {'ME-ref'} is enough. Andy02124 (talk) 16:51, 26 August 2024 (UTC)

It should just be:

['CITEREFTolkien2017'] = {'ME-ref'},

The B&L isn't part of the template name. -- LCU ActivelyDisinterested «@» °∆t° 20:34, 26 August 2024 (UTC)

Big ole whitelist request

Template:Cite Cambridge History of China (which I've had no involvement in yet) is needing the module-level whitelisting to avoid miscategorisation. There are (to date) seventeen volumes, so there are a lot of things.

@Renata3: can you add (*deep breath*):

['CITEREFLoeweShaughnessy1999'] = {'Cite Cambridge History of China'}, ['CITEREFDienKnapp2019'] = {'Cite Cambridge History of China'}, ['CITEREFPeterson2016'] = {'Cite Cambridge History of China'}, ['CITEREFChaffeeTwitchett2015'] = {'Cite Cambridge History of China'}, ['CITEREFTwitchettSmith2009'] = {'Cite Cambridge History of China'}, ['CITEREFPeterson2002'] = {'Cite Cambridge History of China'}, ['CITEREFTwitchettMote1998'] = {'Cite Cambridge History of China'}, ['CITEREFMoteTwitchett1998'] = {'Cite Cambridge History of China'}, ['CITEREFFrankeTwitchett1994'] = {'Cite Cambridge History of China'}, ['CITEREFMacFarquharFairbank1991'] = {'Cite Cambridge History of China'}, ['CITEREFMacFarquharFairbank1987'] = {'Cite Cambridge History of China'}, ['CITEREFTwitchettLoewe1986'] = {'Cite Cambridge History of China'}, ['CITEREFFairbankFeuerwerker1986'] = {'Cite Cambridge History of China'}, ['CITEREFFairbank1983'] = {'Cite Cambridge History of China'}, ['CITEREFFairbankLiu1980'] = {'Cite Cambridge History of China'}, ['CITEREFTwitchett1979'] = {'Cite Cambridge History of China'}, ['CITEREFFairbank1978'] = {'Cite Cambridge History of China'}

Thanks in advance and sorry. (Noting for clarity that "TwitchettMote1998" alongside "MoteTwitchett1998" is not an error) Folly Mox (talk) 14:58, 19 October 2024 (UTC)

Oh actually it turns out I'm the most recent contributor to that template 🫠 I added a parameter two months ago and forgot all about it. Anyway though Folly Mox (talk) 17:01, 19 October 2024 (UTC)

Sure,  Done. Renata•3 00:41, 20 October 2024 (UTC)

Making whitelisting simpler and more efficient

In the sandbox, I simplified and accelerated the whitelisting code. To remind everyone: wrapper templates can fail the "does footnote target exist" check in the main module at line 36.

My changes (in Module:Footnotes/sandbox, Module:Footnotes/whitelist/sandbox, Module:Footnotes/anchor_id_list/sandbox):

1. I made the whitelisting process only have two alternatives: either an explicit check on the anchor id, or a Lua pattern check on the anchor id (conditional on the corresponding wrapper template being in the article)
2. I made the Lua pattern check much more efficient
   • Upon loading the pages, Module:Footnotes/anchor_id_list computes an intersection between all templates on the page and all wrapper templates that require Lua pattern check
   • This intersection is likely empty for the large majority of articles: no per-anchor overhead will be incurred on those articles.
   • When the intersection is non-empty, each anchor id is scanned only for those patterns that are relevant for wrapper templates actually used on the page. This minimizes pattern matching overhead.
3. Module:Footnotes/whitelist had data structures for doing per-volume whitelisting. As far a I could tell, that was not being used anywhere. I converted all of that checking to either per-anchor-id whitelisting or Lua pattern whitelisting. This should reduce the number of false positives in Category:Harv and Sfn no-target errors.

I'm hoping some other editor(s) will look at my changes before I check them into the main Module. The tests at Module:Footnotes/testcases mostly pass. As far as I can tell, even before my changes, the test_sfn tests were all marked as failing: I manually checked them and they seem fine now. I suspect it's because of UNIQ string mismatch.

Pinging @Trappist the monk who wrote a lot of this code.

Thanks in advance for any comments or suggestions! — hike395 (talk) 17:35, 28 November 2024 (UTC)

What I wrote was likely junk because it was a response to immediate needs of the community so it's good that you've improved and simplified. I'm not going to expend any effort trying to figure out what you've done. If it works, great; if it doesn't, you broke it, you fix it. You might want to make sure that Module:Footnotes/whitelist/sort will still work correctly after you implement your changes.

—Trappist the monk (talk) 18:36, 28 November 2024 (UTC)

I accept "you broke it, you fix it". I will check Module:Footnotes/whitelist/sort. Thanks for the feedback! — hike395 (talk) 18:43, 28 November 2024 (UTC)

Later:

• I made some end-to-end tests of Module:Footnotes/sandbox at Template:Sfnp/test 1 and Template:Sfnp/test 2. The module appears to work in real situations.
• To simplify the system, I'm assuming that using wrapper templates will not generate incorrect refs. That is, a wrapper template should know its own target id. There should be no need to generate an anchor id with that target id: it should never be "no target".

I will wait 2 more days before I move sandbox over to main, in case someone notices something or objects. — hike395 (talk) 23:14, 29 November 2024 (UTC)

The clock is ticking?

First bullet point. To know if the new module is actually working, I would expect to see no target and multiple target error messages; prove that what was working still works. There are none because the test pages are in the template namespace. I don't remember why that constraint was added; perhaps it was discussed on an associated talk page.

Second bullet point makes no sense to me. What is it that you are really saying? Examples?

—Trappist the monk (talk) 00:30, 30 November 2024 (UTC)

1. If you look at line 92 of Module:Footnotes/sandbox, I added logging of the "no target" errors, independent of namespace. The log is placed at the bottom of the HTML output, so you can check to see whether target_check is working. For Template:Sfnp/test 1, the module correctly does not produce any "no target" errors. For Template:Sfnp/test 2, the module correctly produces a single "no target" error. I can add a third test for "multiple target" errors (my changes shouldn't have affected that). Oh, shoot. I see what Trappist is saying. The code for target_check is suppressed for namespace=10 (Template). I will fix.
   • Later -- moved the tests to the Template talk namespace (leaving behind redirects), so the tests will work now. — hike395 (talk) 17:48, 30 November 2024 (UTC)
2. I don't think I explained it at all well. First, let me define some terms for clarity, to make sure we are understanding each other.
   • Let's call a "reference" a list item produced by, e.g., {{cite web}} that shows up in, e.g., a References section. Such references can have ids (generated by, e.g., |ref=) to be pointed to.
   • Let's call a "footnote" an item produced by, e.g., {{sfn}} or {{harvnb}}, that shows up in, e.g., a Notes section. These footnotes each point to a reference by hyperlinking to the reference id.
   • A "no target" error is when a footnote has an id which does not correspond to a reference id.
   • IIUC, A list of reference ids is generated by Module:Footnotes/anchor id list by scraping the article contents and storing them in article_whitelist
   • IIUC, A list of footnote ids is generated by the core function in lines 198-209 of Module:Footnotes/sandbox
   • A problem occurs when there is a wrapper template, because the reference ids generated by those is non-obvious.
   • To remedy the problem, the sandbox code checks the following:
     • At lines 74-82 of Module:Footnotes/sandbox, the footnote id is checked against a whitelist of footnote ids: if a corresponding template is used in the article, then we assume that the template generated the footnote id, and no error is thrown. (This is the same as the live code).
     • At lines 84-88 of Module:Footnotes/sandbox, the footnote id is checked against one or more Lua patterns (which get populated at lines 805-821 of Module:Footnotes/anchor id list, if certain templates are used in the article. If the footnote id matches the Lua pattern, it does not throw an error.
   • What I was trying to say above is that we (usually) do not need to populate article_whitelist with reference ids generated by wrapper templates. The checking for wrapper templates should be completely handled by lines 74-88 (as described above) and does not require an entry in article_whitelist
   • The only case where article_whitelist must be populated is if the reference id is generated explicitly by the call to the wrapper template, e.g., if |ref= is set.
   • Assuming that this reasoning is correct, I simplified the logic of anchor_id_make_wrapper() in Module:Footnotes/anchor id list/sandbox at line 502: when |ref= for a wrapper template is missing or "harv", don't generate a reference id to place into article_whitelist.
3. We can wait longer than 2 days if Trappist (or anyone else) wants to take a closer look at the edits. Hope this helps! — hike395 (talk) 08:16, 30 November 2024 (UTC)

I am going to promote sandbox to main now. I will watch the following categories:

• Category:Pages with script errors (426) (currently 445, not expecting an increase)
• Category:Harv and Sfn no-target errors (4,807) (currently 4,767, expecting a modest decrease from eliminating false positives)
• Category:Harv and Sfn multiple-target errors (7) (currently 1, not expecting an increase).

If anyone sees anything odd, please ping me and I (other other templateeditors/admins) can revert. Thanks in advance! — hike395 (talk) 22:29, 1 December 2024 (UTC)

I saw that {{EB1911}} (along with other templates) did require parsing to generate the correct reference id, so I restored that code to Module:Footnotes/anchor id list. Now there is redundant checking for template wrapping, I moved {{Cite bryonames}}, {{Cite NatureServe}}, and {{PLANTS}} out of the Lua pattern checking and turned them into wrapper templates (whose reference id is inferred). Everything should work now. — hike395 (talk) 06:29, 2 December 2024 (UTC)