Reality Arbitrage Pipeline compares what the crowd seems to believe with what Kalshi is pricing. It scrapes public narrative, converts that into a probability, matches the closest live market, and scores the gap as a “reality distortion.” APIs/services used: Kalshi Trade API for live markets, contracts, and prices Apify API for scraping apify/google-search-scraper for Google News apidojo/tweet-scraper for X Reddit public JSON endpoints for subreddit/search scraping Contextual AI Generate API for narrative probability and final insight generation OpenAI API for embeddings and fallback text generation Redis Cloud / RediSearch for vector storage and nearest-market lookup Google News RSS as a fallback when Apify is not used Optional/in-repo but not the current demo default: Polymarket Gamma API support still exists in the codebase, but the current demo path is Kalshi-focused.
