Research talk:Identification of Unsourced Statements/Citation Reason Pilot

Unclear

The text on the image is unreadable, but the test seems to miss many aspects. Things I'm likely to consider when deciding whether a statements needs a citation: 1) bibliography or other sources in the nearby text; 2) personal knowledge about the topic/statement and comparison to general knowledge thereof; 3) presence of a citation in the same paragraph; 4) content of the cited material and comparison to the text carrying the citation; 5) other sections; 6) talk page. --Nemo 18:57, 18 December 2017 (UTC)

• Having worked through one batch today, I wouldn't disagree with the above. In particular, assessment really needs a visible link to the original article so that the sentence can be assessed in its context. For example, one in the batch was a sentence in what I traced to be part of the en:Æthelwold_ætheling article: the sentence highlighted was citing one commentator, which is covered under Jayakumar, Shashi (2008) in the Sources section of the article. AllyD (talk) 12:41, 23 April 2018 (UTC)

• It looks like this has changed now -- I'm getting the whole article loaded in my screen. Agree philosophically though a bibliography shouldn't alleviate the need for an inline citation as well, right? If we want things to be cited specifically to their source? Unfortunately citations can have variable relationships - cite the sentence, cite the para, cite the whole article... -- phoebe | talk 15:44, 30 April 2018 (UTC)

also it looks like some items in the workset actually do have inline citations, viewable if you go to the whole article, which I assume are to help as a check / with training data. -- phoebe | talk 15:48, 30 April 2018 (UTC)

• Thanks for your comments. After a long thought process, we decided that in this pilot we will remove the majority of the contextual information from the article. This is to check the extent to which the content of the sentence can drive citation behavior. We will monitor the labeling campaign, and if this condition is making the task too difficoult, we will make improvements to the labeling interface, so that it can allow you to make judgements in context, or add textual annotations to your judgements (JADE-style) and/or. We might also proceed with interviews or surveys to understand better how editors make decisions about whether a citation is needed. Miriam (WMF) (talk)

Unable to complete dataset because of bug.

I'm unable to complete the dataset because I'm unable to navigate between the different statements (and the data is bad so don't use it!) When I click on the green segment it doesn't seem to reliably load each segment and associated field.

To reproduce:

Click Save and Next.

Then go back by clicking on the green area of the previous question.

Edit, then click save and next.

Now it fails to load the next section each time this is done.

Mvolz (talk) 14:45, 23 April 2018 (UTC)

"undefined" text selected

It seems I got a broken task in my bundle, on page William Etty. Screenshot here. Excerpt from the developer console:

Object { id: "379011", lang: "en", paragraph_index: 14, revision: "828220623", section: "Success and controversy (1824–35)", section_index: 10, sentence_index: 2, statement: "While some artists disliked the requirement to produce a significant work for no material reward, Etty took the task of illustrating his ability and style extremely seriously, and Sleeping Nymph and Satyrs combines his distinctive attributes of rich colours, pastiche of Poussin, Reynolds and the Old Masters, and nudes painted from life", tid: "f9b7a9b3-2b1e-11e8-885d-d83bd6d2ed93", title: "William_Etty", … }

Hope this helps!

Surlycyborg (talk)

Sorry about that, and thanks for flagging this (and for the screenshot)! We are fixing our input and will correct these kinds of issues. Miriam (WMF) (talk)

Unsourced statements bugs/comments

The tool won't load at all in Safari version 11.1. I ran through a set in Chrome okay, but many of the statements were full paragraphs rather then sentences. I think many of the sentences within these paragraphs should have had their own independent citations. I found it difficult to think of sentences that belong to an article but do not need citations. Perhaps I have not done enough textual editing to be a good labeler..I9606 (talk) 21:49, 28 April 2018 (UTC)

controlled vocabulary for why things need to be sourced

How about a dropdown of reasons for why something might need to be sourced, and then an additional field for other/more? Would save time and would make the data a LOT more consistent from user to user. --phoebe | talk 15:41, 30 April 2018 (UTC)

Thanks for this feedback phoebe! For the first step of this pilot, we would like to collect reasons in free-form text. We will then analyse them to get a reasonable set of pre-defined reason you will be able to chose from using a drop-down menu, in our second pilot. We thought that this way we could ensure that the list of reasons is actually reflecting the citation behavior of editors. See [1] Miriam (WMF) (talk)

undefined

I got an ".....undefined" highlighted in one case: after returning to the tab from another when I had not saved, the text had changed from one article (underwater diving) to another (about Madagascar). No highlighted content found in another example. · · · Peter (Southwood) ^(talk): 05:46, 23 August 2018 (UTC)