#

🔍 Wikipedia

The AskNews Wikipedia search endpoint enables natural language search across the entire Wikipedia corpus, returning relevant chunks, with intelligent diversification, and contextual neighboring chunks to give your LLM a full picture of the most relevant Wikipedia information.

The underlying database is updated monthly with the latest Wikipedia dumps, so you you can count on the latest information in your results.

We have fully open-sourced the underlying structured Qdrant database. You can self host this by heading to our HuggingFace.

The API reference provides the most detailed information

#

Searching Wikipedia

The web search endpoint takes your query, which would be similar to any typical Google search:

You can also control the diversity of the results and if you want to attach the main section to each of your chunks:

{
  "query": "quantum computing",
  "n_documents": 10,
  "neighbor_chunks": 2,
  "diversify": 0.7,
  "include_main_section": true
}

#

Diversifying results

This query will return the most relevant chunks with their 2 neighbors before and 2 neighbors after attached. The diversity parameter at 0.7 pushes diversity close to the maximum, 1.0, and the include_main_section parameter attaches the main section of the article to the start of each chunk, providing additional context to your LLM.

#

Getting full articles

You can also get the full articles instead of the relevant chunks. You can do this by setting the full_articles parameter to true. This overrides the neighbor_chunks and include_main_section parameters.

{
  "query": "quantum computing",
  "n_documents": 10,
  "full_articles": true,
  "diversify": 0.7
}

Parameters may evolve, the most updated reference is always the API reference.

#

Availability

The wiki endpoint costs 1 Fast API call, for details on how many calls are included in your plan, please head to the rate limiting page.