FOMC Intelligence
NLP pipeline turning Federal Reserve communications into market signals.
Graduate Research Assistant, Stony Brook University
- 200K+
- data points processed
- 3.7K+
- time-series intervals
- <300ms
- P99 API latency
- 27%
- simulated portfolio return
The problem
Federal Reserve language moves markets, but the signal is buried in hedged prose. The interesting question is not what the FOMC said — it is what changed since the last statement, and whether that delta is priced in.
Approach
Treat each statement, minutes release, and speech as a document in a time series. Extract sentiment and stance, diff against the prior release, and align the result to market data on the same intervals.
Architecture
- Ingestion: AWS Lambda pulls Fed publications and financial news on a schedule, landing raw documents in S3 with schema validation at the boundary.
- Processing: transformer-based sentiment and stance extraction, plus statement-over-statement diffing to isolate what actually changed.
- Storage: DynamoDB holds structured per-interval features across 3.7K+ time-series intervals.
- Serving: FastAPI exposes the feature store to downstream consumers at sub-300ms P99.
- Observability: Prometheus and CloudWatch track ingestion lag and pipeline failures.
Design decisions
Event-driven ingestion over a monolithic batch job
Fed publications arrive irregularly and the market reaction window is short. Lambda triggers keep end-to-end lag low and make partial failures recoverable per-document instead of per-run.
Diff against prior release rather than scoring in isolation
Absolute sentiment on central-bank prose is nearly constant — it is engineered to be. Almost all the information is in the change, so the pipeline is built around comparison, not classification.
Schema validation at ingestion, not at read time
Upstream formats change without warning. Failing loudly at the boundary keeps malformed documents out of the feature store, where they would be far more expensive to detect.
What was hard
- Backtests are trivially self-deceiving. Any signal computed with information unavailable at decision time produces spectacular and meaningless returns, so the pipeline enforces point-in-time correctness on every feature.
- Sustaining zero data loss under bursty load when several publications land at once.
- Fed language is deliberately hedged; off-the-shelf sentiment models trained on consumer text score it as uniformly neutral and had to be adapted.
What I took from it
- The modeling was the easy part. Correct point-in-time data plumbing was most of the work and all of the risk.
- Reported returns from a simulation are a statement about the backtest, not about the market — and the site says so rather than implying a track record.
Stack
Other work
- Product
Atriveo
Job-search platform with a 5.0-rated Chrome extension and live customers.
- AI Systems
Legal RAG
Retrieval-augmented generation over legal filings, with citations that hold up.
- Machine Learning
MRI Tumor Viewer
Brain tumor segmentation running entirely in the browser.