SShortSingh.
TopTechnologyAI & MLStartupsProgrammingScienceBusinessFinanceCrypto & Web3World
Back to feed
Programming

Developer finds 73.5% win-rate FX system loses money due to flawed backtest logic

0
·2 views

A software developer auditing a live foreign exchange trading system built over 16 years discovered it was losing money despite a 73.5% win rate across nearly 5,000 out-of-sample trades. The core flaw was that the backtest subtracted transaction costs from final profit-and-loss totals rather than embedding them into trade entry and exit triggers, causing the system to count winning trades that would never have won in live conditions. Compounding the problem, the system used small profit targets relative to stop-losses, mathematically requiring a win rate above 75–82% just to break even. A walk-forward parameter analysis across 119 folds returned near-zero rank correlation between training and test performance, indicating no learnable signal existed in the data. The author concluded that testing whether a parameter surface is learnable at all should precede any optimisation effort, and that breakeven transaction cost is a more practical edge-detection metric than Sharpe ratio.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

How one dev team built a single AI tutor routing 60+ features across five subjects

A development team built a multi-subject AI tutoring platform covering general academics, Indian competitive exams like JEE and NEET, language learning up to CEFR C1, career coaching, and an arena mode. Rather than letting each feature manage its own model calls, the team centralized all AI-backed features into a single registry with defined pricing, model chains, and fallback logic. A cost audit revealed that 'thinking tokens' were the biggest expense, prompting a full architectural redesign of the routing layer. The team also discovered a critical flaw where mastery scores never updated because quiz triggers relied on undocumented user-typed keywords, which was fixed by adding explicit UI controls and a shared progress engine. Key engineering lessons included treating response caching as an allowlist, issuing automatic refunds when cheaper fallback models respond, and ensuring errors surface visibly to users rather than silently rendering blank screens.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Developer Builds AI Memory Agent With Intentional Forgetting, Benchmarks It Against Naive Systems

A developer built an AI memory agent called Synapse for the Global AI Hackathon Series on Qwen Cloud, designed to selectively forget outdated information rather than storing everything indefinitely. Unlike conventional approaches that treat all data equally in a vector database, Synapse assigns each memory a decaying salience score based on importance, recall frequency, and elapsed time. The system uses two different decay rates — around 72 hours for episodic details and 30 days for stable facts like preferences — so it forgets casual conversation while retaining meaningful information. A background process periodically consolidates repeated mentions into single memories and detects contradictions, automatically retiring stale facts when newer ones supersede them. The developer benchmarked the system against a naive vector-store approach using real Qwen Cloud API calls throughout, with no mocked responses, to validate the claims in practice.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Engineering team cuts RAG pipeline latency 40% using Bayesian search and hybrid retrieval

A development team rebuilt their retrieval-augmented generation (RAG) pipeline from scratch after standard production deployments proved inadequate for legal, API, and support-ticket use cases. The team replaced fixed 512-token chunking with document-type-specific strategies, including recursive, semantic, and agentic chunking, achieving recall@10 scores between 91% and 97%. They combined vector search with BM25 and cross-encoder reranking using Reciprocal Rank Fusion, which improved relevance correlation from roughly 0.75 to 0.92. A Bayesian-informed search optimization was also applied, ultimately reducing overall query latency by 40%. The team published their configurations and Python code to help other engineers replicate the tunable retrieval pipeline.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

How One Team Built Automated LLM Evaluation That Catches 92% of Hallucinations

A development team discovered critical gaps in their AI quality process after deploying a RAG-based customer support assistant that served hallucinated responses to over 500 users before the issue was detected. The failures included fabricated billing policies and rate-limit figures pulled from a competitor's documentation, exposing the team's reliance on informal, manual testing. In response, they built a production-grade evaluation pipeline featuring a suite of automated judges assessing faithfulness, instruction-following, schema validity, safety, and domain accuracy. The system integrates directly into CI/CD workflows, blocking code merges that degrade output quality and enabling regression detection in real time. The team reports the pipeline now catches 92% of hallucinations before deployment, replacing subjective human review with structured, metric-driven evaluation.

0 comments Read more at DEV Community