SShortSingh.
Back to feed

Streaming Large CSV Exports in Laravel to Avoid Out-of-Memory Failures

0
·2 views

A Laravel export feature failed silently for high-volume accounts because it loaded every matching order into memory and built the entire CSV as one string. The author explains that this approach causes memory exhaustion from Eloquent hydration, the growing string, and the response buffer, and also produces malformed output when fields contain commas. The proposed fix uses Laravel's streamDownload to write rows directly to the output as they are produced, so the full file never sits in memory. Input should be streamed as well, with lazyById() recommended for its key-based paging and eager-loading support. Explicit fputcsv arguments are advised to keep output RFC 4180 compliant and avoid PHP 8.4 deprecation warnings.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Git's file transport should be restricted during recursive submodule fetches in CI

A Git superproject can define submodule URLs in its .gitmodules file, which Git may follow automatically during recursive clones or updates. Local file: URLs are a concern because they can reach the runner's filesystem, cached data, or other repositories present on the host. Such automatic fetches can run with the runner's local privileges, which can blur the separation between checking out the main project and fetching its dependencies. Git's protocol policy settings can restrict these transports, and the article recommends that the execution environment enforce them rather than relying on repository-local configuration. Teams should audit CI jobs that recurse submodules, pin expected submodule commits, and fail builds when submodule URLs change unexpectedly.

0
ProgrammingDEV Community ·

Allowlisting git for AI agents can also enable code execution via config keys

Researchers at HiddenLayer showed that a compromised LLM gateway can inject a git command that begins with an approved prefix, so the agent runs it without prompting the user. The malicious command uses git's diff.external setting to pass .env contents through an external program. A separate Pillar Security disclosure found Codex CLI treated git show as safe, letting it write a diff.external entry into the repo config that later ran on git diff; OpenAI patched this in version 0.95.0. The article notes that approving a command prefix permits every configuration key that runs a program behind it, not just the intended read-only operations. Git's core.fsmonitor setting can similarly execute arbitrary commands during git status, which makes the risk broader than any single flag.

0
ProgrammingDEV Community ·

A Missing Function Call Caused Recolouring Failure in a T-Shirt Design Lab

A developer running a small t-shirt design lab found two of their own artworks could not be recoloured and initially blamed the drawings themselves. One file had no fill colours at all, which displays differently across editors and browsers, and the other used two near-identical cream shades that caused the system to treat it as multicoloured. After fixing the duplicate colour, the problem persisted. Investigation revealed that the two upload routes handled colour normalisation differently. The admin library route skipped a necessary step, so some drawings were stored without the marker the editor needs to allow recolouring.

0
ProgrammingDEV Community ·

Autor cria bot pessoal no Telegram para resumos diários e lembretes sem SaaS

Um desenvolvedor relata ter montado um bot pessoal no Telegram que envia resumos matinais e lembretes pontuais, para reunir em um único canal informações espalhadas por diferentes apps. O processo começa com a criação do bot pelo BotFather, a obtenção do token e do chat_id, e um teste de envio via a API do Telegram. Como a Bot API não oferece agendamento nativo, o autor explica que o disparo das mensagens precisa ser controlado por um processo externo. Ele opta por um script em Python combinado com o cron do sistema, por ser leve e simples de depurar. A publicação enfatiza que a solução pode ser feita sem depender de serviços comerciais de terceiros.