The MDA Search Engine combines four complementary retrieval layers to ensure comprehensive accuracy across 35,455 state files and 199,998 curated document chunks:
1. Dense Semantic Vector Search (ChromaDB)
Embeds queries into 384-dimensional dense vectors to match concepts, context, and meaning across 199,998 high-signal passages (e.g. matching "renewable energy funding" to "ARRA clean power allocations").
2. BM25 Lexical Ranking (Okapi BM25)
Scans tokenized corpus of 35,455 documents for exact keyword matches, rare technical terms, vendor names, and document titles.
3. MDA Entity Pattern Inverted Index
Catalogs structured state codes across 10,880 files: grant numbers (MS-EE...), dollar amounts ($12.5M), fiscal years (FY2024), percentages, and statutory code sections (§ 57-39-19).
4. Neural Cross-Encoder Re-Ranking (ms-marco-MiniLM-L-6-v2)
Evaluates query and candidate excerpts jointly through a neural transformer, assigning fine-grained relevance scores and selecting only the top highest-confidence excerpts for the AI model.