How Delta Tables Work Inside Microsoft Fabric's Lakehouse Storage

·1 views

Delta tables are the default table format in Microsoft Fabric's Lakehouse, built on Parquet files and enhanced with a transaction log stored in a dedicated _delta_log folder. This transaction log records every insert, update, delete, and schema change as a sequential JSON file, enabling features like ACID transactions, time travel, and schema enforcement. Each Delta table in the Lakehouse is physically stored under the Tables folder in OneLake, Fabric's unified ADLS Gen2-compatible storage layer, as a collection of compressed columnar Parquet files read in parallel by Spark. Unlike plain Parquet files, Delta tables support actual row-level updates and deletes, making them far more suitable for analytical workloads. Tables created via Spark, pipelines, or Dataflows are automatically registered and become queryable through the Lakehouse's SQL endpoint without manual configuration.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

New Claude-based tool 'text-lens' analyzes writing without rewriting or suggesting edits

A developer has built a Claude-powered writing tool called /text-lens, designed to reflect what a piece of writing is doing rather than rewrite or improve it for the author. Unlike tools such as Grammarly or Sudowrite, text-lens does not suggest replacements or generate new content; instead, it identifies specific moments in a text and explains what a reader experiences there. The tool first determines the genre of the submitted text — poem, argument, narrative, etc. — before applying a tailored analytical lens, since different text types have fundamentally different structural concerns. Analysis is governed by 11 internal rules intended to prevent the AI from shifting into a tutoring or ghostwriting role. The underlying premise is that writers struggle not from a lack of skill but from a perceptual limitation: they read what they intended to write, not what is actually on the page.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer shares layered security guide after accidentally leaking database password on GitHub

A developer building a personal side-project discovered their database password had been hard-coded and committed to a public GitHub repository, prompting an urgent cleanup of the codebase. The incident led them to research and combine three core security practices: storing secrets in environment variables or cloud secret managers instead of source code, using Let's Encrypt with Certbot for automated TLS certificate management, and configuring firewalls to deny all traffic by default except explicitly required ports. Rather than treating these as separate tasks, the developer reframed them as interconnected layers of a unified defense strategy. The resulting guide includes before-and-after code examples in Python and Nginx to illustrate each fix in practical terms. The key takeaway is that even hobby projects carry real security risks and benefit from the same foundational protections used in production systems.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer Builds Free Adaptive IQ Test Using ML and Next.js 14

A developer built and launched IQ Platform, a free adaptive cognitive assessment tool, after growing frustrated with the static nature of most online IQ tests. Unlike standard tests that present identical questions to all users, the platform calibrates question difficulty based on a user's age, education level, and occupation. It covers six cognitive domains — numerical, verbal, pattern, logical, memory, and spatial — with questions rated on a difficulty scale from 1 to 5. Scoring is handled by a custom machine learning regression model built without external ML libraries, running entirely on the client side with no backend or database. Session history is stored via localStorage, preventing users from encountering repeated questions across multiple attempts.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer Builds Minimal 'mintOS' Using C, Assembly, GRUB, and QEMU

A developer has published a hands-on guide detailing how to build a basic custom operating system called mintOS from scratch. The project uses C and x86 assembly to write a kernel that displays text on screen via VGA text-mode memory at address 0xB8000. GRUB serves as the bootloader, reading a grub.cfg configuration file to load the compiled kernel binary into RAM at startup. Supporting tools include GCC for compiling C code, a linker script to arrange the kernel in memory, and QEMU to emulate and test the OS virtually. The result is a minimal bootable OS that prints a static message, demonstrating the full boot chain from BIOS/UEFI through GRUB to the running kernel.

0 comments Read more at DEV Community