SShortSingh.
Back to feed

Node.js Upload Storage: Keep Files in PostgreSQL or Move Bytes to Object Storage?

0
·2 views

The article argues that small, bounded files can reasonably live in PostgreSQL alongside their metadata within one transaction, while larger upload workloads often benefit from storing immutable bytes in S3-compatible object storage. It stresses that the cheapest option is whichever backup, restore, deletion, and failure handling a team can realistically operate, not merely the lowest storage price. The author recommends a split design, with metadata in PostgreSQL and opaque object keys pointing to the bytes, once uploads could meaningfully affect database queries or recovery. Across either design, the application must remain the authorization boundary, and deletion should be traceable and physically executed through a retryable process. The piece also warns that HTTP caching can expose private documents and recommends choosing Cache-Control directives based on sensitivity and verifying headers at the edge.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Company Names Do Not Establish Identity: 14 Tests Before CRM Import

Prospect files may describe a legal enterprise, one of its establishments, a trading name, a corporate group, or a contact, so similar names alone are unreliable for matching records. Importers that treat name similarity as proof can wrongly merge branches into parents, attach people to the wrong account, or erase separate buyers. The article recommends matching only on evidence that identifies the same entity, routing relationships and ambiguity to review, and never treating a score as certainty. It explains that French identifiers distinguish legal units (SIREN) from establishments (SIRET), so records sharing a SIREN may still describe different locations. The author offers a synthetic 14-case test pack and a conservative triage approach, with identity decided before person-level deduplication.

0
ProgrammingDEV Community ·

Dev.to API rate-limit retries cleared only half the time in one developer's pipeline

A developer's comment-reply pipeline hit dev.to API 429 rate-limit errors ten times since August, according to its work log. Counting outcomes showed that in-run retries cleared the limit in five cases and failed in five, with the failures resolved only by later scheduled runs. The script contains no retry or sleep logic; each backoff interval was improvised by whichever run encountered the error and then discarded. The author argues the log cannot distinguish whether retries or simply elapsed time caused the successful recoveries.

0
ProgrammingDEV Community ·

Connection Pools Starve When AI Agents Hold Leases During Slow External Calls

An article on DEV Community argues that outages can occur even when database CPU is nearly idle, because connections are held open while waiting on external systems. The author notes that AI agent runtimes, which leave a pooled connection leased during slow LLM or API streams, are a common cause. Using Little's Law, the piece shows that a 20-connection pool handling 2ms queries can sustain about 10,000 requests per second, but drops to roughly 0.7 requests per second when each connection is held for 28 seconds. The result is queuing, timeouts, and HTTP 500 or 503 errors despite low resource utilization. The article argues that vertical scaling does not solve the problem, and that the same pattern affects Redis, HTTP clients, Kafka, and CRM API quotas. It also states that single-threaded unit tests and staging environments typically fail to surface this issue.

0
ProgrammingDEV Community ·

Forwarded Headers Behind Tunnels: Why X-Forwarded-For, Proto and Host Can't Be Trusted Blindly

When an application runs behind a tunnel, headers such as X-Forwarded-For, X-Forwarded-Proto and X-Forwarded-Host determine what the app believes about the caller, the protocol and the requested hostname. Misconfiguring trust can cause harmless problems like redirect loops or a rate limiter treating all visitors as one, or more serious ones where an attacker controls values that security checks rely on. Tunnel providers such as ngrok and Cloudflare append client information rather than sanitizing it, so client-supplied values remain in the header. Only entries added by proxies the developer controls should be trusted, and reading the leftmost value is unsafe for security decisions. Redirect loops occur when an app enforcing HTTPS sees plain HTTP because TLS terminates at the edge, and frameworks such as Django and Rails provide proxy-header settings to resolve this.