The proliferation of disinformation poses a significant threat to democratic processes, particularly during elections. Newsrooms, especially in regions like Sub-Saharan Africa, often find themselves overwhelmed by the sheer volume of false information, lacking the real-time tools and resources to effectively combat it. This challenge was the driving force behind the Center for Collaborative Investigative Journalism's (CCIJ) development of ElectionWatch, an innovative AI-powered system designed to empower journalists in their fight against election-related disinformation.

The problem: Navigating a sea of election disinformation

CCIJ's journey to ElectionWatch began with a year-long investigation into Nigeria's 2023 presidential elections. This extensive work exposed a chaotic landscape of data manipulation, including doctored ballot boxes and fraudulent image uploads. What became clear was the immense amount of manual effort required to sift through this "craziness in the data," says Nelly Kalu, CCIJ’s Editorial Project and Product Manager. The team recognised that much of this labor could be automated and that their experience offered valuable lessons for other journalists.

CCIJ's ElectionWatch originated from an investigation into Nigeria's 2023 elections, revealing widespread data manipulation. This highlighted the need for automation to manage the sheer amount of anomalous data". The core problem CCIJ aimed to solve was assisting journalists with disinformation investigations during elections. They expanded the definition of disinformation to include "malinformation" and malicious intent. Their goal for ElectionWatch was to strengthen election reporting and protect democracy from information weaponisation by tracking traditional falsehoods combined with malicious intent.

Building the solution: ElectionWatch takes shape

The solution to this pressing problem was an AI-powered system that functions as an AI agent. CCIJ deliberately chose this approach to avoid the risk of hallucination and misinterpretation, ensuring that the tool delivers precise, fact-based analysis. ElectionWatch analyses disinformation, maps out the involved actors and their networks, and creates timelines of events. This empowers journalists to quickly grasp narratives, identify key players, understand the chronology, and ultimately produce impactful reporting that counters disinformation. It shifts the focus from basic fact-checking to a comprehensive analysis of disinformation's real-time effects.

After securing a grant from JournalismAI, CCIJ embarked on the development journey. They began with extensive consultations, discussing the tool's design and data gathering strategies during elections. They met with various organisations, which had amassed a wealth of data from the Nigerian elections. These conversations helped define the scope of data collection, with a particular focus on scraping difficult-to-access platforms like Telegram and TikTok.

CCIJ then hired a UX/UI designer to map out the user journey. Through user research, their Minimum Viable Product (MVP) addressed 11 out of 13 identified pain points and needs of their target users: fact-checkers, journalists in Sub-Saharan Africa and small newsrooms globally. Our vision for ElectionWatch is to be a global tool used not only in the global south but newsrooms globally who may need its service. The emphasis was on creating a simple, step-by-step tool that provides useful analysis. Visual appeal was also crucial; CCIJ wanted ElectionWatch to be a "companion" that engaged journalists, making their work feel less like an additional burden.

Collaborations were vital. Through the extensive JournalismAI community, they connected with experts who guided the brainstorming and contributed to data analysis from TikTok and Telegram. For the DevOps aspect, CCIJ partnered with WebWorks, an African team whose understanding of regional nuances and data complexities, coupled with their willingness to follow CCIJ's lead, made them an ideal choice, says Kalu.

The technical build used Python, MongoDB Atlas, and Google Cloud services for the core stack. Machine learning and AI involved the Google ADK, Open Router, spaCy, scikit-learn, and various transformers to pre-train LLMs for disinformation classification, tokenisation, and entity/relationship recognition. Collaboration tools included GitHub, Slack, WhatsApp, Discord, Notion, and Basecamp. The front-end and visualisation relied on HTML, JavaScript, D3.js, and Plotly. Testing and quality assurance used pytest, Postman, and Cypress.

A significant aspect of the development was the integration of local languages. Recognising the importance of nuance, CCIJ hired journalist-translators for Igbo, Hausa, Yoruba, and Nigerian Pidgin English. This ensured the LLMs were trained to detect these specific linguistic nuances, a challenging but crucial step for the Kenyan WebWorks team. The team also worked with a couple volunteers to crack the TikTok and Telegram challenge. One of the volunteers came from Factiverse.

CCIJ structured their team to work in an agile manner, establishing a four-month build timeline. The CCIJ team focused on product and project management, data analytics, editing, and data development and editorials, while the WebWorks team provided technical expertise, including a technical project manager, technical lead, API data engineers, machine learning engineers and interns, backend and frontend developers, and designers. Editorial leads handled narrative framing, translation, and reporting, with a QA lead and community testers ensuring quality. The team's agility and commitment were paramount, especially given the rapid development cycle.

The opportunities: Expanding ElectionWatch's reach and impact

The future holds exciting opportunities for ElectionWatch. CCIJ’s immediate goals include enabling real-time scraping and live fact-checking. This means highlighting existing fact-checks around disinformation, rather than generating new content, to avoid the risks of hallucination. They also aim to strengthen the governance and ethical frameworks surrounding the tool and enhance its real-time interaction capabilities with journalists.

A key development is the creation of a robust "data engine" – a cleaner, more deliverable, and referenceable data backbone for newsrooms worldwide. This engine will incorporate metadata, timestamps, and geotags. CCIJ also plans to develop an AI newsroom kit, evolving their methodology into a comprehensive "how-to" guide and offering training for newsrooms on using ElectionWatch.

While mindful of the risks, CCIJ is exploring the potential for generative AI at the very last stage, only for summarising completed analysis, emphasising their commitment to avoiding unrequested or hallucinated content. The ultimate goal is to transform ElectionWatch from a data analyst collector into a full-fledged, automated tool that robustly supports journalists.

Lessons for newsrooms

• Embrace humility and iteration: Approach data, disinformation, and machine learning with humility and a willingness to learn. Adopt an iterative approach, unafraid of mistakes, by accepting to "Let it be ugly... Let's see that it's not working" to speed up development.

• Recognise linguistic nuance "translators": Acknowledge the crucial need for specialised experts ("translators") to bridge the gap between technical teams and local context to accurately capture linguistic nuances in research and data.

• Cultivate active inquisitiveness: Product managers must be actively inquisitive, striving to understand why things fail even in areas outside their primary expertise. Attention to detail is vital for effective problem-solving.

• Maximise budget for future funding: Newsrooms, especially those with limited resources, should maximise the demonstrated potential of their current investment to clearly justify and secure future funding.

• Foster partner and team belief: Belief in the product among partners (like WebWorks and Factiverse) is essential for deep investment. Small teams should leverage all available tools, learning resources, and partnerships to compete and create impactful tools, proving that a small team can build a global product.