On Wednesday, Wikimedia Deutschland announced a new database that will make Wikipedia\u2019s wealth of knowledge more accessible to AI models.<\/p>\n
Called the Wikidata Embedding Project, the system applies a vector-based semantic search \u2014 a technique that helps computers understand the meaning and relationships between words \u2014 to the existing data on Wikipedia and its sister platforms, consisting of nearly 120 million entries. <\/p>\n
Combined with new support for the Model Context Protocol (MCP), a standard that helps AI systems communicate with data sources, the project makes the data more accessible to natural language queries from LLMs.<\/p>\n
The project was undertaken by Wikimedia\u2019s German branch in collaboration with the neural search company Jina.AI and DataStax, a real-time training-data company owned by IBM.<\/p>\n
Wikidata has offered machine-readable data from Wikimedia properties for years, but the pre-existing tools only allowed for keyword searches and SPARQL queries, a specialized query language. The new system will work better with retrieval-augmented generation (RAG) systems that allow AI models to pull in external information, giving developers a chance to ground their models in knowledge verified by Wikipedia editors.<\/p>\n
The data is also structured to provide crucial semantic context. Querying the database for the word \u201cscientist,\u201d<\/a> for instance, will produce lists of prominent nuclear scientists as well as scientists who worked at Bell Labs. There are also translations of the word \u201cscientist\u201d into different languages, a Wikimedia-cleared image of scientists at work, and extrapolations to related concepts like \u201cresearcher\u201d and \u201cscholar.\u201d<\/p>\n The database is publicly accessible on Toolforge<\/a>. Wikidata is also hosting a webinar for interested developers<\/a> on October 9th.<\/p>\n Techcrunch event<\/p>\n \n\t\t\t\t\t\t\t\t\tSan Francisco The new project comes as AI developers are scrambling for high-quality data sources that can be used to fine-tune models. The training systems themselves have become more sophisticated \u2014 often assembled as complex training environments<\/a> rather than simple datasets \u2014 but they still require closely curated data to function well. For deployments that require high accuracy, the need for reliable data is particularly urgent, and while some might look down on Wikipedia, its data is significantly more fact-oriented than catchall datasets like the Common Crawl<\/a>, which is a massive collection of web pages scraped from across the internet.<\/p>\n
\n\t\t\t\t\t\t\t\t\t\t\t\t\t|
\n\t\t\t\t\t\t\t\t\t\t\t\t\tOctober 27-29, 2025\n\t\t\t\t\t\t\t<\/p>\n