Hey, It Works! Logo
Hey, It Works!

Tech Blog by Daniel Rosehill

How Are DeepSeek's Innate Current Information Retrieval Capabilities?

How Are DeepSeek's Innate Current Information Retrieval Capabilities?

Deep Seek has been all over the news for the past few days with the AI community in awe of its abilities to rival OpenAI on major benchmarks - for a fraction of the development cost.

Before getting too excited, though, it's worth considering how these tools can be accessed.

Firstly: Navigating The Deep Seek API Nomenclature

Having been using Deep Seek for a number of months now (according to one X poster that makes me a “hardcore geek!”), one of the first things to know about it is that the API naming is a little confusing.

With the world of AI experiencing seismic shifts every few days, the expression “at the time of writing” has never been more useful than at the moment. So with that disclaimer aside, at the time of writing (the publication date of this blog) here is why everybody is getting excited about these models:

Deep Seek has opted for a very different approach to OpenAI when naming models.

While users of ChatGPT are shielded from these strange details, here's what the model selection on OpenAI looks like currently (via Open Router).

There are actually dozens of different models available:

And even relatively famous variants like 4.0 actually have different timestamped variants which users can access:

The two models that Deep Seek provides through its API actually track different (actual) models on the backend: deepseek-reasoner maps onto R1 and deepseek-chat maps onto V3. In my opinion this is actually a wise decision and means that users effectively only have to pick one or the other when choosing the model for their prompt.

Who Is The President Of The USA, AI Tool?

Given that the White House has just had a change of guard It's currently a great time to use my go-to test prompt for assessing whether models appear to have a built-in RAG pipeline. Sometimes these are undisclosed or mysterious forces at work in the background providing LLMs with knowledge that cannot be explained through their training data alone.

Here's the first test prompt I ran. Note that v3, confusingly, self-describes its training data cutoff as being in October 2023 (running the same prompt reveals that the model varies a bit in its description).

Trying out R1 via Open Router we get the same result, with the model this time stating its training date cutoff to be July:

Note how ChatGPT responds (ChatGPT is an OpenAI API model as well as the famous platform):

Sonnet 3.5 actually does worse, claiming decisively that Biden is their current president without mentioning the constraint of its knowledge:

Augmenting the model with real-time search capabilities via Tavily, however, finally yielded an accurate answer, although performance was a little sluggish:

Or via DeepSeek’s web platform:

Conclusions

  • While Deep Seek's cost-performance ratio is astounding, it nevertheless needs augmentation in order to retrieve real-time and recent data.