HFresh, Engram, and AutoIndex

Hello Weaviate Community! 🤗

HFresh brings memory-efficient vector search to workloads where lower heap usage and dataset scale matter more than peak throughput. We also take a closer look at Engram's approach to maintaining agent memory, show how late-interaction retrieval handles charts and tables in PDFs, and explore AutoIndex with Sam O'Nuallain on the podcast.

Latest AI & tech insights

Explore our recent Weaviate content:

Read

• 🔖 How to extract meaning from charts and tables in PDFs: See how late-interaction multi-vector retrieval embeds each PDF page visually, preserving charts, tables, and layout without OCR, text extraction, or chunking. The post includes NVIDIA earnings-deck queries, Query Agent answers with page-image citations, and a deployment-ready Python pipeline. Read the blog

• 💾 HFresh: Memory-Efficient Vector Search: This technical deep dive explains how HFresh combines an in-memory routing layer with disk-based postings and incremental background maintenance for large, mutable datasets. Read the blog

• ✍️ Building Foundry Part 3: From archive to creative search: Follow Foundry as it scans a 23-asset creative archive without moving the source files, builds a manifest, and synchronizes enriched records with Weaviate. Deterministic IDs prevent duplicates, while keyword, semantic, and hybrid search help find assets despite inconsistent filenames. Read the blog

Watch

• 🎙 AutoIndex with Sam O'Nuallain - Weaviate Podcast #143!: Sam O'Nuallain explains how AutoIndex uses analysis and code agents to write representation programs that chunk, enrich, and reorganize data for better Recall and nDCG. The conversation covers validation-gated optimization, BM25's diagnostic transparency, and examples from the CRUMB benchmark. Watch the full podcast

• 🧠 Stop dumping chat history into your context window: See why accumulating and retrieving raw chat messages can overload context with outdated or contradictory facts, and how Engram actively maintains agent memory instead. The video also walks through setting up a basic chat application with Engram. Watch the video

Product highlights

Check out our latest product updates.

• HFresh vector index (GA): Run vector search with lower heap usage by keeping a compact routing layer in memory and storing vector postings on disk.

• Two-stage retrieval: An in-memory centroid index identifies relevant regions before HFresh fetches and searches only the corresponding disk-based postings.

• Incremental maintenance: Background splitting, merging, and reassignment keep postings balanced as data changes, without disruptive full-index rebuilds.

See the full HFresh blog for the details.

Hungry for more?

Have a question or want to connect? Join our Weaviate Forum to engage in community conversations.

Follow our Blog, LinkedIn, Twitter, and YouTube for more content updates.

See you in two weeks,

Shan