Overusing Broadcast Joins Can Trigger Spark OutOfMemory Failures

·2 views

A popular quick fix for slow Spark shuffles is forcing a broadcast join, which can work initially but becomes risky as the broadcast table grows. Symptoms include stalled executors, heartbeat timeouts, heap-space OutOfMemoryErrors during serialization, and high GC time in the Spark UI. Adding cluster nodes does not help when a single broadcast table exceeds per-executor memory. The article notes that the default auto-broadcast threshold of 10 MB can be misleading, since on-disk file size can inflate several times in memory. It recommends checking the BroadcastExchange operator in the SQL plan, verifying the smaller table is the one being broadcast, and adjusting or disabling the threshold where needed.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

NeuroBrix uses V100 tensor cores via Triton kernels, enabled only where benchmarks show gains

Many V100 GPUs in universities and cloud instances go underused because Triton compiles matrix products on Volta to ordinary CUDA-core instructions, leaving the tensor cores idle. NeuroBrix's nbxtensor library adds hand-written kernels built on the mma.sync.m8n8k4 instruction for key operations such as matrix products, convolutions, flash attention and fused mixture-of-experts GEMMs. These kernels compete with the portable Triton kernels in a certification process that checks correctness against a float64 reference and times each candidate on the target GPU class. The engine then serves whichever path the certificate records as faster for each operation and shape. In a November 2026 benchmark on Tesla V100 hardware, the Triton engine outperformed PyTorch in 30 of 36 model-and-card pairs, reaching up to 5.24x on Janus-Pro-7B, while being slower in the remaining six.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Read-only MCP server lets AI agents diagnose TeamCity build failures

A new 1.0.0 release of a read-only MCP server for TeamCity, published November 2, lets an AI agent analyze failed builds from a pasted link. The agent pulls error lines from the build log, lists failed tests with new failures flagged first, and identifies the commits included in the build. The server exposes 29 tools covering projects, builds, logs, tests, agents and the queue, but cannot trigger, cancel, pin or tag anything. It can also locate build configurations from approximate descriptions, asking the user to choose when candidates are similarly plausible. The author recommends issuing the token with the Read-only preset and a per-project limit to match that restriction.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer Builds Python TUI Log Debugger to Replace grep and tail

A developer grew frustrated with locating errors in long production logs using grep and tail, so they built an open-source terminal tool called tracetui. The application is built on the Textual framework and organizes logs into three panels for log sources, a color-coded stream, and detailed trace views. Errors are highlighted in red, and the tool automatically detects loguru, standard logging, and plain-text formats. It is released under the MIT license and can be installed via pip, with log files passed as command-line arguments.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Anthropic buys $3.4B in gas turbines, taking ownership of power generation capacity

Anthropic reportedly agreed in late September to pay about $3.4 billion for 1.7 gigawatts of heavy-duty gas-fired generators from San Antonio supplier OnePWR Solutions, according to the Financial Times. The deal reportedly took over equipment orders that OnePWR had placed about two years earlier but could not finance, along with the engineering and construction capability to install them. Neither Anthropic nor OnePWR commented, and Reuters said it could not independently verify the report, though investment bank Piper Sandler lists a $3.4 billion sale of OnePWR's power generation platform to a frontier AI lab. The purchase works out to roughly $2,000 per kilowatt and is small relative to Anthropic's roughly $895 billion in disclosed commitments. Analysts note the move appears aimed at securing scarce place in a turbine and grid-connection backlog that stretches into the early 2030s.

0 comments Read more at DEV Community