Same Prompt, Different Output: Testing Five Reproducibility Myths in Free-Model Pipelines

·2 views

A developer questions common assumptions that model endpoints are deterministic, retries are harmless, and a 200 response means work ran only once. They provide a stdlib-only Python harness with a deliberately nondeterministic stub server to demonstrate variation even at temperature 0. The test runner logs two-phase receipts, comparing output hashes across identical runs. The author argues that rather than promising sameness, pipelines should store request keys, output fingerprints, attempt IDs, status, and timestamps to detect drift and duplicate side effects.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Spark Liquidity Layer Report Flags High Oracle Manipulation Risk Across $2.7B TVL

A security report dated 27 October 2026 assigns Spark Liquidity Layer, a multi-chain liquidity hub spanning Ethereum and its layer-2 networks, an overall oracle manipulation risk score of 8 out of 10. The protocol's pricing relies on an aggregation of oracles, including Chainlink, Uniswap V3 and Curve TWAPs, a Synthetix spot oracle, an internal liquidity-weighted oracle, and an off-chain API feed. That price data drives LP token minting and withdrawals, cross-chain bridge rates, and collateral calculations for the Spark Margin product. The report identifies manipulation of the internal oracle through concentrated capital in low-liquidity pools as a medium-to-high likelihood vector, and front-running around TWAP snapshots as another. It also flags the off-chain feed as a single point of failure that could be compromised or spoofed.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Restaurant POS Access Control Should Govern Actions, Not Job Titles

Cashiers, kitchen staff, managers, and owners all need different levels of access to a point-of-sale system. The article argues that a role name alone is not a sound security rule, proposing instead that permissions represent specific capabilities such as creating orders, managing tables, or processing refunds. Roles are presented as bundles of these permissions, which can be individually adjusted to handle real-world exceptions. The author stresses that the backend must enforce these checks on every protected action, since hiding interface buttons does not prevent unauthorized access. Finally, the piece cautions that overly granular permission lists can become hard to understand, recommending clearly named permissions and auditable changes.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer Recovers Abandoned Checkouts by Tracking Stripe Sessions Directly

A developer building OutfitMaker, an AI outfit planner, found that only one of eleven recent checkout attempts was completed. Their recovery email missed most abandoned checkouts because it only searched subscription records, which could not capture people who opened a one-time pass purchase. The fix was a new table that logs every Stripe Checkout session with its plan and completion status. A recovery job now runs every 30 minutes and emails people whose checkouts were opened 60 to 180 minutes earlier. The email reopens the exact offer through a one-click link instead of sending the person back to the pricing page.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Former dispatcher builds MorPro software for small trucking fleets from family business

Yamil Morales started as a dispatcher for his father's Miami trucking business in 2012, where he saw how much operations depended on phone calls, spreadsheets and memory. In 2014 he founded MorPro and began building software for his own growing fleet, since existing tools targeted large carriers or brokers. The company's platform, first used by its flagship fleet Pathway Transportation, now covers dispatch, fleet maintenance, compliance monitoring and AI agents for operations, finance and compliance. MorPro became a full technology company in 2022 and moved its base from Miami to Los Angeles. Morales says the best product guidance came from working alongside the people who run trucks day to day.

0 comments Read more at DEV Community