SShortSingh.
Back to feed

How to run n8n in production using queue mode to prevent workflow failures

0
·1 views

Self-hosted n8n commonly fails in production when heavy workflow executions overload the single process that also serves the UI and handles webhooks, causing editor freezes and silent workflow stops. A second common culprit is an unbounded executions table in Postgres that grows until it becomes a disk or performance bottleneck. N8n's recommended fix is queue mode, which splits the application into a main instance that manages the UI and webhooks and one or more worker instances that pull and execute jobs from a Redis queue. This separation means a heavy execution cannot block incoming webhooks, and workers can be scaled horizontally since all state is stored in Postgres. Key hardening steps include enabling execution pruning to cap history, scheduling verified database backups, pinning the n8n image version, and keeping Postgres and Redis off the public internet.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Infra Lang v0.5.1 Lets Developers Define Infrastructure Once, Deploy Across Multiple Platforms

A developer has released Infra Lang v0.5.1, an open-source Infrastructure-as-Code compiler designed to eliminate repetitive configuration across tools like Kubernetes, Docker Compose, Helm, and Terraform. The tool allows users to write infrastructure definitions once in a typed domain-specific language and compile them to multiple target formats. Version 0.5.1 introduces features including service dependency management, cloud secret store integration, a network policy security DSL, and FinOps cost estimation with CI/CD guardrails. The cost estimation feature can generate Markdown reports for pull requests and automatically fail pipelines if projected monthly costs exceed a set budget. The project is open source and available on GitHub at github.com/TuviDev/infra-lang, with installation possible via pip.

0
ProgrammingDEV Community ·

ByteChef Launches Managed RAG Knowledge Base with Vector Search and OCR Support

ByteChef has introduced a built-in Knowledge Base feature that lets teams store and semantically search documents using retrieval-augmented generation (RAG). Users can upload files in formats including PDF, Markdown, Word, JSON, and plain text, with scanned documents handled via an OCR pipeline. The platform automatically parses, chunks, embeds, and indexes content into a pgvector-backed PostgreSQL database, with configurable chunk sizes and overlap settings per knowledge base. Retrieval is accessible through AI Agent tools, workflow actions, and a built-in search interface for testing queries before deployment. ByteChef manages the vector schema and indexing automatically, though users remain responsible for scheduling re-ingestion and manually handling upstream document deletions that the system cannot detect on its own.

0
ProgrammingDEV Community ·

Bitaxe Naja Duo Becomes First Open-Source Bitcoin Miner to Break 10 J/TH Barrier

The Bitaxe Naja Duo, set to ship next week, is the first open-source home Bitcoin miner to achieve sub-10 joules per terahash efficiency, running at 9.5 J/TH at stock settings. It is built around two BM1373 chips — Bitmain's 3nm SHA-256 ASICs from the Antminer S23 generation — which carry no public datasheet and required community reverse-engineering to operate. An ESP32-S3 microcontroller runs the open-source esp-miner firmware, handling chip initialisation, work distribution, nonce collection, and a self-hosted web interface called AxeOS. The dual-chip design introduced per-chip temperature monitoring and a 2-phase voltage regulator to manage heat and power delivery more precisely than earlier single-sensor boards. All performance metrics are accessible via a REST API, and the firmware is released under GPL-3.0, requiring any shipped derivatives to publish their source code.

0
ProgrammingDEV Community ·

Agentic Engineering: The Structured Approach Replacing Vibe Coding at Scale

In early 2025, AI pioneer Andrej Karpathy popularized 'Vibe Coding,' a method of building software by describing ideas in natural language and letting AI generate the code, which proved effective for quick prototypes but struggled at scale. As projects grow, developers encounter what practitioners call the '200-prompt spiral,' where fixing one bug cascades into multiple new ones due to poorly understood architecture. Agentic Engineering has emerged as the next step, defined as the disciplined practice of architecting, scoping, and directing AI agents through decomposed prompts, strict typing, and test-driven feedback loops. The approach categorizes AI tools into three tiers — rapid all-in-one builders, AI code editors for surgical precision, and autonomous terminal-level agents — recommending a hybrid workflow across them. Practitioners advocate that classical software engineering fundamentals, such as modular design, ubiquitous language documentation, and TDD, become more critical than ever when AI handles code generation.