Karsten Wenzlaff, Advisor
August 26th, 2025
Karsten Wenzlaff is a Crowdfunding and Fintech Expert from Germany. He is Secretary-General of Digit[...]
Musk Says Human Data for AI Training is Depleted
AI Data | Jan 10, 2025
Image courtesy of AI
AI Hits a Data Wall. What Does This Mean for Fintech and Investors?
AI's data feedback loop was predicted a couple years ago to end in 'model collapse'. Well, as reported in a Guardian article, Elon Musk just boldly stated that artificial intelligence systems have already exhausted all human-generated (publicly available) data available for training -- and it happened last year. Now, AI developers are facing a shortage which might push the industry towards alternative data sources like synthetic (AI generates on its own) and alternative approaches. The implications of this shift could be massive, given that fintechs depend on AI for a lot of heavy lifting including everything from fraud detection to trading algorithms. So, what does this mean?
The Data Wall Problem
The limitation of available training source data can limit innovation and create sector bottlenecks in applications, so exhausting earth's human generated data supply is a real problem. Competition in AI is fierce globally, and data-hungry AI models are consuming data faster than ever. Without access to fresh data, AI models could stagnate or lose their edge in detecting fraud, creating personalized customer experiences, or even identifying new insights and market opportunities. The idea of 'model collapse' is when the quality of data degrades to the point where too many errors are introduced, amplifying the problems, and skewing the truth with too many 'hallucinations'. AI systems that are trained on AI outputs can get caught in a feedback loop where models lose accuracy and diversity over time, resulting in bias and the propagation of misinformation. There's also an issue with disproportionately relying on 'minority data' (less popular data or non-mainstream data) to train models, it can distort the truth.
Synthetic Data and Alternatives
More and more developers and AI experts are turning to synthetic data despite the risks described above. When there's a lack of training data, AI can create its own synthetic data to train itself by running simulated scenarios that might not appear often in real life. For examples, a fintech company may use synthetic data to mimic rare fraud patterns or test unusual market conditions. Models that rely on too much synthetic data can generate inaccurate or nonsensical outputs that can have serious implications, especially in finance where trust and accuracy is paramount.
See: Will Nuclear Fuel the Data-Driven Future?
Developers are on the hunt for untapped, new sources of human generated data. Data exchange marketplaces are becoming more popular as a result. Amazon Web Services, for example, runs the AWS Data Exchange, that allows companies to securely access and use third party datasets for AI training and analytics including sensitive information like financial data or healthcare. Another option to check out is the Snowflake Data Marketplaces which is part of Snowflake's Data Cloud. Its a platform that offers live, ready to query datasets across finance, consumer behaviour, and public sector data.
Depending on the application and AI model, developers can turn to niche or specialized data sources such as physical records that are being digitized, such as books, manuscripts or other archives. Cultural data sources from difference regions or countries could be used as inputs for AI training data. Behavioural data can be collected from trading/investment activity or interactions with websites, apps, and games. Data can also be generated from real-time sources such as IoT devices with sensors in smart homes, cars, and smart cities. Data can be collected in real-time from Human-AI interactions too, such as when customer support bots or conversational tools interact with humans. Project owners can also incentivize and crowdsource data online, depending on their requirements.
Strategic Shifts in AI Development
1. Multi-modal models. AI systems are moving beyond just text and are incorporating more diverse data sources such as visual (charts, infographics, satellite imagery etc), audio (voice patterns, phone calls, meetings), and transactional data (i.e., payment trends). See: Google’s Vision for a Real-Time, Multimodal AI Assistant
2. Smaller, specialized models. Rather than be the top leading Large Language AI model for all things, that require massive training data, innovators are creating AI systems for specific tasks. These smaller solutions require less data while still delivering the desired results. These small language models are also better for the environment as they consume less energy.
See: RBC and Cohere Partner on ‘North for Banking’ AI Platform
3. Efficiency over scale. AI researchers are prioritizing was to improve their AI models by making them more efficient by interpreting the data they have in a way that squeezes more insights from less information.
4. Blended training approaches. This strategy combines synthetic with real-life data in a balanced way to reduce risks and errors. See NVIDIA, Google, OpenAI example.
5. Federated learning. A way of training AI models that across multiple devices/servers that store and process data locally, instead of sending that data to a central server. This improves data privacy and security, and the raw data generated never leaves the locally stored location.
Outlook
The data shortage will spur innovative approaches and create new investment opportunities in synthetic data generation, AI privacy tools, and alternative data collection methods. At the same time, AI companies that rely too much on synthetic data or outdated models could become less accurate with reduced performance, so caution is required. Most companies will need to use data more wisely, explore new data sources, and specialized strategies to compete.
The National Crowdfunding & Fintech Association (NCFA Canada) is a financial innovation ecosystem that provides education, market intelligence, industry stewardship, networking and funding opportunities and services to thousands of community members and works closely with industry, government, partners and affiliates to create a vibrant and innovative fintech and funding industry in Canada. Decentralized and distributed, NCFA is engaged with global stakeholders and helps incubate projects and investment in fintech, alternative finance, crowdfunding, peer-to-peer finance, payments, digital assets and tokens, artificial intelligence, blockchain, cryptocurrency, regtech, and insurtech sectors. Join Canada's Fintech & Funding Community today FREE! Or become a contributing member and get perks. For more information, please visit: www.ncfacanada.org
 |  |  |
|---|---|---|
 |  |  |
Leave a Reply