Fact Checking Methodology

Before its operation around the middle of February 2024, MultiFactChecking (MFC) had established a standardized fact-checking methodology. This includes, for instance, researching claims, verifying sources, and providing clear verdicts (e.g., true, false, misleading, and unproven). In verifying or evaluating sources, for instance, the staff of MFC will prioritize credible and verifiable sources for fact-checking and avoid using dubious or unverified sources. The methodology of verifying claims looks as stated in the following points:

• Information or claims for fact-checking will be collected from various sources, including news articles, social media posts, government, and non-government reports, and so on.
• Editorial team discusses in their meeting how best to verify claims and give directions.
• MFC does not evaluate the accuracy of ideas, opinions, philosophies, or arguments. We evaluate the accuracy of what people say and present as factual claims. We focus on both the exact words they use and the context of the statement.
• The assigned fact-checker verifies claims by reaching out to the original source and by using various technological tools.
• The Editorial team discusses the publication of the verified claims and gives direction for writing up and publication of the articles.

The structure of and elements in writing up article takes the following procedures:

The structure starts with giving the lead of what was CLAIMED and right away informing the reader about its VERDICT (this will be discussed at length below). It then gives a bit more detail into what has been claimed, who claimed it, how many followers/subscribers does a person who made the claim on social media has, how many times claims were shared and received likes. Using this as foundation, the article then goes deeper in analyzing and giving recommendations on how people could counter such mis/disinformation by themselves using verification tools and/or reaching out to the original source of the claim. Finally, the article gives context or background to the claim. Side note: we are now considering adding at the end of each article how readers could reach out to MFC in case there will be correction to be made in the articles and how they inform MFC in case there are claims they wanted to be verified (see MFC’s correction policy below).

Thematically, the MFC – as indicated in its name – covers multiple areas including political and socio-economic issues. Because, considering the context of Ethiopia, mis/disinformation in politics is highly intertwined with socio-economic affairs and vice versa. For example, verifying political claims has advantages to the day-to-day socio-economic activities of people in Ethiopia and verifying socio-economic claims have the similar effect on political activities of people.

Generally, MFC prioritizes reaching out to the ORIGINAL SOURCE of the claim and tries to strike it while the claim is viral and before it causes further mis/disinformation that leads to conflict. MFC will also update reports as new information becomes available. Therefore, MFC makes a proper attribution to sources and provides references to support claims and conclusions. Avoiding ad hominem attacks, respecting privacy, and copyright and working on public interest issues are at the core of MFC’s ethical guidelines.

On Languages

MFC has been working with four languages that include Afaan Oromo, Amharic, Tigrigna and English to reach its larger and multiple audience. Initially, Fact-checkers presented their preferred languages for fact-checking. However, all pitch claims that they wanted to verify in English – the working language of MFC. Then, the verified article, for instance, written in Afaan Oromo will be translated to Amharic, if the claim made in Oromia affects Amhara region. For example: If one claims Fano militiamen from Amhara region attacked villages bordering Oromia region and this was pitched by MFC’s Afaan Oromo Fact-checker, then the article will be produced in Afaan Oromo and translated to Amharic.

One of the reasons MFC does translations was due to financial and human resource capacity. Because the MFC just started its operation with limited finance that limits recruiting more fact-checkers. The other reason for translation is, of course, due to the severity of conflicts on this border and MFC’s mission to inform the public and counter mis/disinformation that contributes to conflicts in Ethiopia.

On Rating Claims

After collection and analysis, the articles get values or get verdict. For this, MFC will use a consistent way of giving verdicts to establish the accuracy of claims. These verdicts are, for instance, True, False, Unproven, and Misleading, – when claims carry partially false statements. MFC understands the difficulties of determining if a statement is false or completely true as there might be some statement left out or a new statement emerges in the future that contradicts the verified content. Therefore, to be in a safe zone, the MFC “[focuses] by necessity on claims that can be directly verified (Is Barack Obama an American citizen? Is the election on Wednesday?) or bolstered by a reasonable consensus of appropriate experts or authorities (Did GDP increase last year?)” (Guess & Lyons, 2020). Thus, rating claims look as follows:   

1. FALSE: Content that has no basis in fact. Therefore, the claim is inaccurate according to the best evidence publicly available at the time, e.g., claim that a natural disaster took place, when no such event happened.
2. MISLEADING: Elements of the claim are accurate but presented in a way that is misleading. This include content that has been edited or synthesized beyond adjustments for clarity or quality, in ways that could mislead people about something that has no basis in fact. For example, adding an image into an authentic photo to present the appearance of something that actually never happened.
3. PARTLY FALSE: Content has some factual inaccuracies. This include, but not limited to, a mix of true and false key claims, where the false claims do not predominate. For example, a list of several claims, some that are true and some that are false.
4. MISSING CONTEXT: Content that implies a false claim without directly stating it. For example, an unaltered video clip of a group chanting that they’re peacefully protesting, but the full version of the video shows the same group instigating violence.
5. SATIRE: Content that uses irony, exaggeration, or absurdity for criticism or awareness, particularly in the context of political, religious, or social issues, but that a reasonable user would not immediately understand to be satirical.
6. UNPROVEN: Evidence publicly available at the time neither proves nor disproves the statement. More research is needed.
7. TRUE: Content that contains no inaccurate or misleading information according to the best evidence publicly available at the time.

Source: Meta and 211Check