gomarc brings MARC21 library data processing to Go, outpacing Python's pymarc by up to 11x

·2 views

A developer has released gomarc v0.1.0, a Go port of the popular Python library pymarc, designed to parse and process MARC21 bibliographic data used in library systems. Benchmarks run on two real catalogue exports totalling over 226,000 records show gomarc completing MARC-8 Unicode parsing up to 11 times faster than its Python counterpart. The library supports reading and writing binary MARC21, MARCXML, and MARC-in-JSON formats, and offers an API familiar to existing pymarc users. Benchmark results were validated for correctness by confirming byte-identical output between both libraries before any performance figures were recorded. The project is at an early v0.1.0 stage, with raw byte decoding and MARC-in-JSON write performance noted as areas still to be optimised.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

How to Self-Host Langfuse LLM Observability Platform Using Docker Compose

Langfuse is an open-source observability platform designed to monitor LLM applications by tracking traces, token usage, costs, and providing debugging analytics for AI workflows. A technical guide published on DEV Community outlines how to deploy Langfuse on a Linux server using Docker Compose, combining PostgreSQL, ClickHouse, Redis, and S3-compatible object storage. The setup is secured with Traefik as a reverse proxy and uses Let's Encrypt for automated TLS certificate management. Deployment requires a minimum of 4 vCPUs and 16GB RAM, a configured domain A record, and six randomly generated secrets for securing database and application credentials. Once running, the platform allows developers to send real traces through the stack and monitor production AI application behaviour from a self-hosted environment.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Controlled Vocabularies, Taxonomies, and Ontologies: Know What You Actually Need

In knowledge management, controlled vocabularies, taxonomies, and ontologies represent three distinct levels of data structuring, each roughly an order of magnitude more complex than the previous. A controlled vocabulary is simply a fixed list of agreed-upon terms with definitions, identifiers, and statuses — enough to enable consistent filtering and reporting. Adding a broader/narrower hierarchy to those terms creates a taxonomy, which enables roll-up queries and faceted navigation but introduces challenges like hierarchy disputes and non-tree-shaped domains. Ontologies go further by defining classes, properties, and logical axioms that allow machines to infer new facts, but require specialized modeling expertise and reasoning infrastructure. The article cautions that the term 'ontology' is frequently misused to describe all three levels, and most teams seeking one actually need nothing more than a well-maintained list of forty agreed-upon terms.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

How to Safely Export and Verify ML Models Using ONNX Runtime

ONNX Runtime allows a single model artifact to run across CPU, GPU, and various accelerators via one API, eliminating much of the per-platform engineering work. However, the export process can silently introduce errors, including frozen control flow, operator decomposition into approximations, and numeric drift. Developers must carefully configure dynamic axes during export to avoid inference failures on variable-sized inputs, and should validate the exported graph's node count as an early warning of inefficient decomposition. Numerical accuracy should be verified by running both the original and exported models on identical inputs and comparing outputs within an explicit tolerance threshold. Opset versioning and provider-level operator support are additional compatibility concerns that can cause parts of a model to run on unintended hardware without any error being raised.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer builds memory auditor for AI agent, finds precision matters more than recall

A developer discovered that their coding agent's markdown-based memory system — spanning 90 files over two weeks — was storing factual claims about file paths, commands, and credentials that could silently become outdated. An automated auditor script was written to verify these claims against the actual machine state, initially flagging 45 of 90 files as invalid. However, the 50% failure rate proved misleading: false positives arose from the script misidentifying prose words, API routes, and Tailwind CSS classes as checkable filesystem or CLI entries. Targeted regex patterns and a CLI allowlist were introduced to distinguish genuinely verifiable claims from contextual or descriptive text. The project highlighted a subtle but critical distinction — a date recording a past event is permanently valid, while a date marking an expiry becomes stale the moment it passes.

0 comments Read more at DEV Community