Weaviate Newsletter - Generative Search is here

Author

Sebastian Witalec
February 14, 2023

Hello Weaviate friends, 🤗

🚀As always, we have a lot to share with you.🔥 The highlight of this newsletter is the Generative Search - Open AI module.⭐️ We announced a couple of meetups (in Boston and Toronto). See you there soon. 🤗⭐️ The CFP for Haystack US is open.⭐️ Weaviate Air episode #5 is ready to watch, and it is time to subscribe for #6.⭐️ Plus, we have a ton of great new content to share.⭐️ Last but not least, we published Weaviate with OpenAI cookbooks.

So, let's get right into it. 🔥

Generative Search – OpenAI

🚀 Last week, we announced Generative OpenAI module (press release), which allows you to use the power of ChatGPT with your data (stored in Weaviate).

Now, you can ask Weaviate to "find documents that talk about x and summarize each in a tweet" – which will find relevant documents and generate a tweet response for each – or "find documents that answer a question and summarize all of them in one response" – which will take the search result as a group and provide a combined response. The opportunities are limitless.

This is a very powerful combo because Weaviate finds the best candidates for the query, while ChatGPT is there to generate a response based on the provided prompt.

ChatGPT for your dataOr in other words[🚀Weaviate🚀] + [🤖Generative OpenAI module🤖] = [🔥ChatGPT for your data🔥].

Example of a query

Here is an example of a query that:

  1. uses vector search to find articles about "Italian food",

  2. picks the top 5 results,

  3. then summarises each response as a Facebook Ad. 


{
  Get {
    Article(
      nearText: {
        concepts: ["Italian food"]
      }
      limit: 5
    ) {
      title
      summary
      _additional {
        generate(
          singleResult: {
            prompt: """
              Describe the following as a Facebook Ad: {summary}
            """
          }
        ) {
          singleResult
          error
        }
      }
    }
  }
}

If you have experience querying Weaviate, then most likely you are already familiar with the first part of the query. The only new thing to learn is the new generate operator inside _additional.

How to get started

The best thing about the Generative OpenAI module is that it doesn't require you to do anything in your schema, vector embeddings, or data. Plus, it will work with any vector embeddings, so there is no special requirement to switch your vectorizer to OpenAI.Requirements

  • Weaviate 1.17.3 or newer

  • text2vec-openai module

    • (For local deployments) – add to your docker configuration – see docs

    • (For WCS) – nothing to add, all new instances will have it enabled by default

  • OpenAI API key

Resources

Meetups and Conferences

We are hosting two meetups in the next few days and we would love to meet you in person 🤗The first meetup is in Toronto – on the 16th, and the next one is in Boston – on the 21st.If you are in the area. RSVP and join us there.

Haystack US – CFP

The CFP for Haystack US is open until the 27th of February.

This is a great place to give your talk about your experience with Vector Search Engines and Weaviate 😉😉

Details:Location: Charlottesville, Virginia, USADates: 25-26 of April – with some extra activities before and after

Weaviate Air

Last week we had another fantastic episode of Weaviate Air.Where we covered topics like:

  • Google Summer of Code

  • The new Generative Open AI module – with two demos

  • Search for weaviate.io – docs, blogs and podcasts

  • Proposal for the new pipe operator

Next episode

The next episode is planned for Wednesday, the 8th of March.

Content

Blogs

  • ✍️ read Deploying a Weaviate cluster on GKE – by Sam Stoelinga – an end-to-end guide that covers: GKE configuration, Weaviate deployment to Helm, and tweaking Helm properties.

  • ✍️ read Weaviate Podcast Search – by Connor Shorten – to learn how Connor used Whisper and Weaviate to search through his podcasts. Audio search? Yes, please.

  • ✍️ read ChatGPT for Generative Search – by Zain Hasan, Erika Cardenas, and Connor – to learn about the new Generative OpenAI module.

  • ✍️ read What to expect from Weaviate in 2023 – by Etienne Dilocker (our CTO) – to learn about the six pillars for Weaviate development in 2023.

Podcast

  • 🎙 (watch and listen) – Generative Search in Weaviate with Bob van Luijt.

Learn Weaviate

In the last few weeks, we collaborated with our friends at OpenAI to create Cookbooks – in the format of Jupyter Notebook – to show how to build projects with Weaviate and OpenAI embeddings.

Check out the GitHub repo with the cookbooks and learn how to:

Thank you for reading

Thank you for your time. I hope you enjoyed this (and all our previous newsletters), and I look forward to meeting you one day 🤗

Thanks,Sebastian