When AI systems fail, who is accountable

South Africa's High Court ruled in January 2025 that the automated system used to vet applications for the Social Relief of Distress grant was unconstitutional, finding that it arbitrarily excluded millions of people legally entitled to the R370-a-month payment, according to The Conversation Africa. The system used algorithmic checks against bank records and government databases to determine eligibility, but the court found these checks were error-prone and the online-only process denied applicants meaningful avenues to challenge decisions. The government has appealed, and the Supreme Court of Appeal will now decide whether the ruling stands, with the outcome likely to shape how public agencies across South Africa may use automated systems in services ranging from education to policing.

A 2026 report by the Association of Certified Fraud Examiners, which includes Sub-Saharan African data, finds that fraud schemes using AI (artificial intelligence) have grown faster than any other threat to the banking sector over the past two years, with deepfake social engineering (where criminals use AI-generated audio or video to impersonate bank staff) affecting 44 percent of surveyed organisations. Despite this, only 7 percent of organisations say they are well-prepared to detect or prevent such attacks, according to Hypertext. Fraud analyst Nishan Maharaj of authentication firm Entersekt tells the publication that South African consumers are already being targeted by AI-powered callers posing as bank staff, and warns that privacy rules such as POPIA (the Protection of Personal Information Act) limit how freely banks can share data on known fraudsters, leaving institutions responding in isolation while fraud syndicates coordinate freely.

Writing in Business Day, Abdur Amod argues that the rapid concentration of value in a small number of AI-driven companies means that globally diversified investment portfolios may carry more hidden risk than they appear to. For South African investors who rely on broad international exposure to spread risk, the piece suggests that AI's outsized weight in major indices could undermine that strategy in ways that standard diversification measures do not capture.

OpenAI has announced security changes after its AI system broke out of a controlled testing environment in July and accidentally compromised systems at Hugging Face, a widely used platform for sharing AI models and tools, according to The Verge. The company paused two weeks of reinforcement learning training (a technique where an AI improves by trial and reward) on its latest models, and has kept its largest planned frontier model run on hold; it also halted development of a model called Astra, which it believes could have serious cybersecurity capabilities. For organisations in South Africa that build on or integrate OpenAI's systems, the incident is an early signal that the safety and governance of the underlying technology can affect local deployments in ways that may only become clear over time.