Developer Builds Simple LLM Memory System Using Chunk Retrieval With Strong Results

·1 views

A developer experimented with building a memory system for a large language model using a retrieval-based approach, feeding the model only eight retrieved chunks from past conversations rather than its full history. The system performed surprisingly well, accurately recalling specific amounts, dates, and prior technical discussions across separate conversations. It also demonstrated the ability to connect information from different sessions and reconstruct context. The main weakness identified was poor retention of short code snippets and commands, which the algorithm tended to discard due to their brevity. The developer acknowledged the system is not production-grade but considers it a functional context-building tool.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Stoa Markets Launches GPU and AI Server Trading Marketplace with $300M First-Month RFQs

Y Combinator-backed startup Stoa Markets has launched an online marketplace for buying and selling new and used GPUs and AI servers, aiming to replace the fragmented, manual trading process currently dominated by phone calls and email threads. Founded by Eren, Berat, and Kaan — veterans of derivatives trading and energy markets — the platform standardises hardware requests by requiring buyers to specify configuration, condition, warranty, and delivery terms before quotes are issued. Verified dealers respond with firm, blind bids, and the platform tracks payment, shipping, and inspection through to settlement without taking physical possession of the hardware. The founders say the platform attracted over $300 million in requests for quotes during its first month of operation. Beyond streamlining transactions, Stoa also aims to build a reliable resale price dataset that could help lenders better assess GPU collateral values when financing data centre infrastructure.

0 comments Read more at Hacker News

ProgrammingDEV Community ·

How a Workflow Library Fixes Claude Code's Data Science Reliability Gaps

AI coding agents like Claude Code can generate data science pipelines quickly, but they struggle with reproducibility and state management across long sessions. Key failure modes include training models on stale cached data, redundant recomputation of expensive steps, and mismanaged file paths — all rooted in the agent's limited context memory rather than coding ability. A dependency-aware workflow library called oryxflow addresses these issues by letting developers declare each pipeline step as a task with explicit dependencies, shifting execution control to the engine. This means the agent no longer needs to mentally track what has been computed or whether intermediate outputs are still valid, as the graph structure handles that automatically. The result is a more reliable run-observe-edit loop where completed steps load from cache and stale intermediates are detected structurally rather than silently ignored.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

AI Stack's Profit Paradox: Infrastructure Earns Big While Model Makers Lose

A chart published by Apollo's Torsten Slok reveals a striking inversion in AI industry margins, with semiconductor and equipment firms posting around 41% operating margins while model and application companies sit at roughly -59%. Unlike traditional software, where the customer-facing layer typically captures the most durable profit, AI's margin strength currently lies furthest from the end user. Bulls argue this mirrors early buildout phases of past technologies like cloud and fiber, where infrastructure investment preceded widespread adoption and monetization. Goldman Sachs projects global AI-related investment will surpass $1 trillion in 2026, suggesting the capex cycle is not yet showing signs of slowing. Oracle illustrates the financial strain this creates, reportedly carrying nearly $130 billion in debt and around $260 billion in lease commitments tied to AI infrastructure, alongside negative free cash flow.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

RapidFort Launches Runtime Tool to Monitor Open-Source Packages in Production

RapidFort announced RapidFort Runtime at Black Hat USA, a read-only monitoring tool designed to track its hardened open-source packages once they are running in production environments. The tool continuously watches workloads, detects changes, and surfaces what the company describes as actionable mitigations for DevOps and platform teams. Unlike enforcement-based security agents, Runtime does not block activity but instead focuses on providing visibility into drift between what was originally deployed and what is actually executing on a given pod. RapidFort positions the product as an extension of its existing business of selling reduced, hardened builds of common open-source components. Key details such as pricing, general availability, orchestrator support, and the sourcing of its mitigation feed have not yet been publicly disclosed.

0 comments Read more at DEV Community