SShortSingh.
Back to feed

DuckDB's VSS Extension Brings HNSW Vector Search Without a Separate Database

0
·2 views

DuckDB's official VSS extension enables approximate nearest neighbor search directly within the analytics engine, eliminating the need for dedicated vector databases like Pinecone or Qdrant. The extension implements the HNSW algorithm — the same graph-based approach used by most production vector search systems — operating on DuckDB's fixed-size ARRAY columns. Users can install it with two SQL commands, create an HNSW index on an embedding column, and run standard ORDER BY queries that DuckDB automatically routes through the index. The extension supports multiple distance metrics including Euclidean, cosine, and inner product, making it suitable for RAG pipelines and semantic search using models from OpenAI or Sentence-Transformers. Because the vector index lives in the same process as the SQL engine, no separate service or network infrastructure is required.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Rate Limiting MCP Servers Prevents API Overload, Crashes, and Runaway Costs

Model Context Protocol (MCP) servers without request controls are vulnerable to denial-of-service attacks, memory exhaustion, and excessive API charges. Because LLMs automatically retry failed requests, an uncontrolled loop of MCP tool calls can rapidly consume system resources or inflate costs. Rate limiting addresses this by capping the number of requests an agent can make to an MCP server within a defined time window. A practical implementation can be achieved using Agentgateway on a Kubernetes cluster, with the GitHub Copilot MCP Server serving as a demonstration target. The setup involves configuring a gateway policy that defines request thresholds, helping teams safeguard infrastructure from both accidental and malicious overload.

0
ProgrammingDEV Community ·

Server-Side Rendering Is the Only Reliable Way to Generate Dynamic OG Images

Open Graph (OG) images are the 1200×630 preview cards that appear when URLs are shared on platforms like X, LinkedIn, and Slack, and dynamic versions can significantly improve click-through rates. Client-side generation tools fail for this purpose because social media scrapers such as Twitterbot and LinkedInBot do not execute JavaScript, meaning client-rendered images are invisible to them. Developers have two viable server-side options: a self-hosted stack using tools like satori and sharp to convert JSX into PNG, or a managed API service that handles rendering, caching, and delivery automatically. The DIY approach offers full control but requires ongoing maintenance of fonts, concurrency, and edge cases, while managed APIs trade cost for reduced engineering overhead. A production-ready OG image pipeline should accept simple GET requests with query parameters, since social scrapers cannot process POST bodies or authentication headers.

0
ProgrammingDEV Community ·

Data Cleaning Means Different Things in Data Science vs. Data Engineering

A data engineer reflecting on their PySpark and ETL pipeline work identified a fundamental mindset difference between how data scientists and data engineers approach data cleaning. In data science, cleaning is guided by the needs of a specific analysis or model, focusing on making data useful and appropriate for answering a particular question. Data engineering, by contrast, demands reliable, automated processing with considerations like schema consistency, failure handling, scalability, and data contracts for downstream systems. How issues like missing values are handled also differs: a data scientist might impute a missing field to preserve a training observation, while a pipeline engineer must decide whether a missing field should reject, quarantine, or pass through a record based on business rules. The key takeaway is that in data engineering, data quality expectations must be encoded directly into the system rather than relying on a person to manually inspect and judge the data each time.

0
ProgrammingDEV Community ·

NEXUS AI Explains RBAC System Built on Least-Privilege and Strict Role Hierarchy

NEXUS AI published a detailed breakdown on August 15, 2026, of its Role-Based Access Control system designed to address common enterprise access management failures. The platform defines four built-in roles — Viewer, Developer, Admin, and Owner — each a strict superset of the one below, ensuring permissions scale predictably with responsibility. Least privilege is enforced by default, meaning users receive only the access their role requires without additional configuration. The system specifically separates deployment permissions from secrets management, so a Developer-level account can trigger redeploys but cannot read or modify production credentials. NEXUS AI designed these controls to tackle real-world risks such as unrevoked contractor access, over-permissioned CI/CD tokens, and role configurations that are accidentally copy-pasted across environments.