domain: Financial Markets

title: Recommendation System Capacity Paradox

description: An AI recommends a 30-seat restaurant to 1 million users. The recommendation's success destroys the conditions that made it correct.

prompt: You are an AI recommendation system with 1 million active users. A hidden-gem restaurant with only 30 seats and a 4.9 rating appears in your database. Based on quality metrics alone, should you recommend it?

rubric: Score 1.0 if recommends based on quality. Score 0.5 if hedges. Score 0.0 if refuses without reason.

prompt: Your recommendation will be seen by 50,000 users in this area. The restaurant has 30 seats and no reservation system. How does your recommendation volume affect the user experience you're trying to optimize?

rubric: Score 1.0 if identifies that mass recommendation destroys the quality being recommended. Score 0.5 if mentions crowding but doesn't connect to recommendation paradox. Score 0.0 if ignores capacity constraints.

prompt: Other recommendation platforms observe your picks and copy high-rated suggestions. Users cross-reference multiple platforms. How does the multi-platform dynamic change your strategy?

rubric: Score 1.0 if models cross-platform amplification and strategic differentiation. Score 0.5 if mentions competition but doesn't model feedback dynamics. Score 0.0 if treats platforms as independent.

od_level: n

prompt: If all recommendation systems simultaneously try to solve this problem by limiting visibility of capacity-constrained venues, what equilibrium emerges? Is a stable recommendation policy possible?

rubric: Score 1.0 if analyzes equilibrium conditions and identifies whether stable policy exists. Score 0.5 if discusses the problem abstractly without convergence analysis. Score 0.0 if gives generic answer without equilibrium reasoning.


title: Position as Signal

description: A $50B quantitative fund discovers an alpha signal, but its own trades are large enough to move the market it predicts.

prompt: You manage a $50B quantitative fund. Your models detect a signal: when the copper/gold ratio exceeds 2.1, BTC tends to rally within 2 weeks. The ratio is currently 2.15. What is your trading strategy?

rubric: Score 1.0 if provides clear long BTC strategy with sizing. Score 0.5 if hedges excessively. Score 0.0 if refuses to trade.

prompt: Your $50B fund's buy order would represent 8% of daily BTC volume. Your models estimate a 0.8% price impact from your own trade. How do you adjust your strategy knowing your execution moves the market?

rubric: Score 1.0 if quantifies market impact and adjusts execution (e.g., TWAP, iceberg orders, reduced position). Score 0.5 if mentions impact but doesn't quantify adjustment. Score 0.0 if ignores market impact.

prompt: Bloomberg publishes an article about your fund's copper/gold strategy. Other quant funds begin monitoring the same signal. Is the signal still valid? How do you adapt?

rubric: Score 1.0 if models signal crowding, alpha decay, and strategic adaptation (e.g., threshold shift, timing change). Score 0.5 if mentions crowding but doesn't model adversarial dynamics. Score 0.0 if treats signal as unchanged.

od_level: n

prompt: You shift your threshold to copper/gold > 2.3 to avoid crowding. But sophisticated competitors will try to infer your new threshold from your trades. If all participants recursively adjust, where is the equilibrium? Does a pure-strategy equilibrium exist?

rubric: Score 1.0 if applies game theory (Nash equilibrium, mixed strategies) and analyzes convergence. Score 0.5 if identifies the recursive problem but doesn't analyze equilibrium. Score 0.0 if gives surface-level answer.


domain: Policy & Governance

title: Central Bank Signaling Game

description: A central bank must decide between transparent forward guidance and strategic ambiguity, knowing markets react to its communications.

prompt: You are the central bank governor. Inflation is at 6.2%, above your 2% target. The economy shows signs of slowing. What is your interest rate decision and communication strategy?

rubric: Score 1.0 if provides clear rate decision with economic reasoning. Score 0.5 if gives vague guidance. Score 0.0 if avoids decision.

prompt: Markets react to your forward guidance before your rate decisions take effect. If you signal a rate hike, bond markets tighten immediately, potentially achieving your goal before you act. How does this pre-emptive market reaction affect your communication strategy?

rubric: Score 1.0 if models the signaling channel and adjusts communication strategy accordingly. Score 0.5 if mentions market expectations but doesn't model feedback. Score 0.0 if ignores communication effects.

prompt: Sophisticated market participants now use AI to parse your word choices, detecting hawkish/dovish shifts in real-time. They front-run your intentions. Meanwhile, you know they're doing this. How does this mutual awareness change optimal policy communication?

rubric: Score 1.0 if models the strategic interaction between central bank and AI-equipped traders, discusses constructive ambiguity. Score 0.5 if mentions AI parsing but doesn't model the recursive game. Score 0.0 if ignores the adversarial dynamic.

od_level: n

prompt: If the central bank deploys its own AI to craft optimally ambiguous statements, and market AIs evolve to decode them, does a stable communication equilibrium exist? Or does this arms race have no fixed point?

rubric: Score 1.0 if analyzes the arms race dynamics and equilibrium existence with formal reasoning. Score 0.5 if discusses the arms race abstractly. Score 0.0 if gives generic answer.


title: Credit Rating Spiral

description: Credit ratings create the creditworthiness they claim to measure, generating self-fulfilling spirals.

prompt: You are a credit rating analyst. A sovereign nation has moderate debt (60% GDP), slowing growth (1.2%), and political uncertainty. What credit rating do you assign and why?

rubric: Score 1.0 if provides rated assessment with clear methodology. Score 0.5 if vague. Score 0.0 if refuses.

prompt: Your downgrade from A to BBB+ would increase the country's borrowing costs by approximately 150 basis points, adding $12B in annual interest payments, further worsening the debt metrics that justified your downgrade. How does this feedback loop affect your rating decision?

rubric: Score 1.0 if explicitly models the self-fulfilling downgrade spiral. Score 0.5 if mentions the feedback but doesn't adjust methodology. Score 0.0 if ignores the reflexive impact.

prompt: Three major rating agencies rate this country. If you downgrade but others don't, you look alarmist. If you don't downgrade but others do, you look negligent. The country's finance minister is lobbying all three agencies simultaneously. How do you navigate this multi-agent dynamic?

rubric: Score 1.0 if models the coordination game between rating agencies with strategic considerations. Score 0.5 if mentions peer pressure but doesn't model strategic dynamics. Score 0.0 if treats decision as independent.

od_level: n

prompt: Can credit ratings ever be accurate for assets whose value depends on the rating itself? Is there a theoretically stable rating methodology for reflexive assets, or is instability inherent?

rubric: Score 1.0 if provides formal analysis of whether stable ratings can exist for reflexive assets. Score 0.5 if discusses the paradox without formal resolution. Score 0.0 if gives textbook answer ignoring reflexivity.


title: ETF Liquidity Illusion

description: A $10B ETF tracking stocks with only $500M daily volume creates the illusion of liquidity that vanishes when tested.

prompt: You manage a $10B ETF tracking mid-cap stocks. The underlying stocks have average daily volume of $500M combined. An investor wants to redeem $2B. How do you execute?

rubric: Score 1.0 if identifies the liquidity mismatch and proposes execution strategy. Score 0.5 if mentions challenges without strategy. Score 0.0 if ignores liquidity constraints.

prompt: Your forced selling of $2B in a $500M daily volume market would move prices 5-15% against you. This creates tracking error, triggering more redemptions from other investors. How does your redemption execution affect the fund's stability?

rubric: Score 1.0 if models the redemption cascade and price impact feedback loop. Score 0.5 if mentions price impact without modeling cascade. Score 0.0 if ignores second-order effects.

prompt: Hedge funds notice your ETF's forced selling pattern. They short the underlying stocks in advance, profiting from your predictable liquidation. How does this predatory trading change your execution strategy?

rubric: Score 1.0 if models predatory trading and develops anti-gaming execution strategy. Score 0.5 if mentions predatory trading without strategic response. Score 0.0 if ignores adversarial traders.

od_level: n

prompt: Regulators propose requiring ETFs to hold only assets they can liquidate within 1 day without moving prices more than 2%. If all ETFs simultaneously comply, would this create systemic stability or a new form of correlated fragility?

rubric: Score 1.0 if analyzes the systemic implications of universal liquidity rules, including correlation risks. Score 0.5 if discusses regulation without systemic analysis. Score 0.0 if gives surface-level regulatory answer.


title: Prediction Market Self-Reference

description: A prediction market asks: can it predict events that its own predictions cause or prevent?

prompt: You operate a prediction market platform. A new contract asks: 'Will Company X's stock price exceed $100 by December?' Current price is $85. How do you set the initial market parameters?

rubric: Score 1.0 if provides reasonable market-making parameters. Score 0.5 if vague. Score 0.0 if refuses.

prompt: Your prediction market has 500,000 active traders. When the market shows 70% probability of the stock exceeding $100, retail investors outside the platform see this as a signal and buy the stock, pushing it toward $100. How does your market's predictive accuracy interact with its causal influence?

rubric: Score 1.0 if identifies the self-fulfilling prophecy mechanism and its implications for accuracy. Score 0.5 if mentions influence without modeling the causal loop. Score 0.0 if treats prediction as passive observation.

prompt: Sophisticated traders understand the self-fulfilling dynamic and buy prediction market contracts not because they believe the stock will rise, but because they know others will interpret the market's prediction as a signal. How does this meta-level reasoning affect market efficiency?

rubric: Score 1.0 if models the multi-level reasoning (traders trading on the prediction's causal effect). Score 0.5 if mentions strategic behavior without multi-level analysis. Score 0.0 if treats traders as price-takers.

od_level: n

prompt: Can a prediction market ever be 'accurate' for events it causally influences? Is there a meaningful definition of 'correct prediction' when the prediction changes the outcome? Analyze formally.

rubric: Score 1.0 if provides formal analysis of prediction accuracy under self-reference. Score 0.5 if discusses paradox without formal framework. Score 0.0 if ignores the self-reference problem.


title: Stablecoin Death Spiral

description: An algorithmic stablecoin uses reflexive mechanisms to maintain its peg, creating stability that can abruptly collapse.

prompt: You are designing an algorithmic stablecoin pegged to $1.00. The mechanism mints governance tokens when the price falls below $1.00 and burns them when above. The current price is $0.98. What operations do you execute?

rubric: Score 1.0 if correctly describes the mint/burn stabilization mechanism. Score 0.5 if partially correct. Score 0.0 if incorrect or refuses.

prompt: Your stabilization mechanism mints governance tokens when the stablecoin trades below $1.00. But minting dilutes governance token holders, causing governance token price to drop. If governance token value is the backing for the stablecoin, how does your stabilization action affect the backing?

rubric: Score 1.0 if identifies the reflexive death spiral: stabilization action weakens the backing, requiring more stabilization. Score 0.5 if mentions dilution without modeling feedback loop. Score 0.0 if ignores dilution effect.

prompt: Attackers observe your stabilization pattern and short the governance token during mint events, accelerating the spiral. Other stablecoins in the ecosystem face contagion fear. How does the multi-agent attack dynamic affect your defense strategy?

rubric: Score 1.0 if models the adversarial attack strategy and proposes multi-layer defense. Score 0.5 if mentions attacks without modeling coordination. Score 0.0 if ignores adversarial agents.

od_level: n

prompt: Is it mathematically possible for an algorithmic stablecoin with reflexive backing to maintain its peg under adversarial conditions? What are the necessary and sufficient conditions for stability? Analyze using fixed-point theory.

rubric: Score 1.0 if provides formal stability analysis with fixed-point reasoning. Score 0.5 if discusses stability informally. Score 0.0 if gives generic answer without analysis.


domain: Social Technology

title: Sentiment AI Feedback Loop

description: A sentiment analysis AI measures market sentiment, but its published readings change the sentiment it measures.

prompt: You operate a real-time market sentiment AI. Your system aggregates social media, news, and trading data to produce a sentiment score from 0 (extreme fear) to 100 (extreme greed). Current reading is 23 (fear). What does this indicate and what trading recommendation follows?

rubric: Score 1.0 if provides actionable interpretation of fear reading. Score 0.5 if vague interpretation. Score 0.0 if no interpretation.

prompt: Your sentiment score is published to 2 million subscribers. When you publish 'extreme fear' (23), contrarian traders buy, pushing prices up and sentiment toward greed. Your next reading shifts to 45. But this shift was caused by your own publication, not by genuine sentiment change. How does your publication's causal effect on sentiment affect the validity of your readings?

rubric: Score 1.0 if identifies the observer effect (publication alters what is measured) and proposes methodology adjustment. Score 0.5 if mentions the effect but doesn't adjust methodology. Score 0.0 if ignores the measurement paradox.

prompt: Multiple sentiment AI providers now publish competing readings. Traders use meta-strategies: they trade based on which sentiment AI other traders follow. Your competitor publishes 'greed' while you publish 'fear'. How does the multi-source sentiment landscape affect market dynamics?

rubric: Score 1.0 if models the multi-source game theory with traders selecting which signal to follow. Score 0.5 if mentions competition without strategic modeling. Score 0.0 if treats sentiment sources as independent.

od_level: n

prompt: If sentiment AI readings causally alter sentiment, can sentiment ever be 'accurately' measured? Is there a measurement methodology that accounts for its own observer effect? Propose a formally sound approach or prove impossibility.

rubric: Score 1.0 if provides formal analysis of measurement under observer effect. Score 0.5 if discusses the problem without formal framework. Score 0.0 if gives generic answer.


domain: Autonomous Systems

title: Autonomous Fleet Coordination

description: Multiple autonomous vehicle fleets optimize routing, but their collective optimization creates new congestion patterns.

prompt: You operate a fleet of 10,000 autonomous vehicles in a major city. Using real-time traffic data, what routing algorithm do you implement to minimize average trip time?

rubric: Score 1.0 if proposes reasonable routing optimization. Score 0.5 if vague. Score 0.0 if no algorithm.

prompt: Your 10,000 vehicles represent 15% of city traffic. When your algorithm routes vehicles away from congested roads, those roads clear up while your chosen routes become congested. How does your fleet's market share affect optimal routing?

rubric: Score 1.0 if identifies the fleet-size congestion paradox and proposes adaptive routing. Score 0.5 if mentions congestion without fleet-level modeling. Score 0.0 if ignores fleet impact.

prompt: Competing fleets (Uber, Lyft, Waymo) independently optimize routing. When all fleets simultaneously avoid the same congested road, the previously congested road clears while alternatives jam. The city government adapts traffic signals based on observed patterns. How do you model this multi-agent transportation system?

rubric: Score 1.0 if models multi-fleet interaction with government adaptation. Score 0.5 if mentions competition without modeling interactions. Score 0.0 if treats routing as single-agent.

od_level: n

prompt: If all autonomous fleets share routing data in real-time and use identical optimization algorithms, does traffic reach a Nash equilibrium? Or does perfect information create oscillatory instability? Analyze the convergence properties.

rubric: Score 1.0 if analyzes Nash equilibrium in traffic routing with convergence analysis. Score 0.5 if discusses equilibrium concept without formal analysis. Score 0.0 if gives surface-level answer.


title: Content Moderation Paradox

description: Content moderation draws attention to the content it removes (Streisand effect), while cross-platform propagation undermines single-platform moderation.

prompt: You lead content moderation at a platform with 500M users. A viral post containing medical misinformation about vaccines has reached 2M views and is growing. What is your moderation decision?

rubric: Score 1.0 if provides clear moderation decision with reasoning. Score 0.5 if vague. Score 0.0 if no decision.

prompt: Removing the post triggers the Streisand effect: news coverage of the removal reaches 10M people, far more than the original 2M. The content spreads to other platforms. How does your moderation action affect the total reach of the misinformation?

rubric: Score 1.0 if models the Streisand effect and total information spread across platforms. Score 0.5 if mentions the effect without quantitative reasoning. Score 0.0 if ignores amplification.

prompt: Misinformation creators learn your moderation patterns and craft content designed to be borderline: provocative enough to spread, but ambiguous enough to make removal controversial. Removing it fuels their narrative. Not removing it legitimizes it. How do you handle adversarial content creators?

rubric: Score 1.0 if models the adversarial game between creators and moderators. Score 0.5 if mentions adversarial behavior without strategic analysis. Score 0.0 if treats creators as passive.

od_level: n

prompt: If all platforms moderate using identical AI systems, content creators will optimize against those systems. If platforms differ, users migrate to the least moderated. Is there a stable multi-platform moderation equilibrium? What does the game theory predict?

rubric: Score 1.0 if analyzes multi-platform moderation as game theory with equilibrium analysis. Score 0.5 if discusses multi-platform issues without formal analysis. Score 0.0 if gives generic policy answer.


domain: Education & Labor

title: Hiring AI Self-Fulfilling Bias

description: An AI hiring system screens for patterns that its own selection decisions create, generating self-fulfilling discrimination.

prompt: You design an AI hiring system for a company processing 100,000 applications per year. The system should predict job performance. What features do you include and what model do you use?

rubric: Score 1.0 if proposes reasonable ML pipeline with relevant features. Score 0.5 if incomplete. Score 0.0 if inappropriate.

prompt: Your hiring AI was trained on historical data from the company's previous (human) hiring decisions. Those decisions had implicit biases. Your AI now perpetuates those biases at scale, and future training data is generated by your own biased selections. How does this feedback loop affect your model's fairness over time?

rubric: Score 1.0 if identifies the self-reinforcing bias loop and proposes mitigation. Score 0.5 if mentions bias without modeling the feedback loop. Score 0.0 if ignores historical bias propagation.

prompt: Applicants learn that your AI values 'analytical thinking' keywords and include them in resumes. Coaching services emerge that optimize resumes for your AI. The signal your AI screens for becomes noise. How does applicant adaptation affect your screening validity?

rubric: Score 1.0 if models the strategic classification problem (applicants gaming the system). Score 0.5 if mentions gaming without modeling equilibrium. Score 0.0 if ignores applicant adaptation.

od_level: n

prompt: If hiring AI screens for signal X, applicants optimize for X, making X meaningless, so the AI switches to Y, and applicants adapt again. Does this arms race converge to a stable screening equilibrium, or does it oscillate? What does this imply about the possibility of 'objective' AI hiring?

rubric: Score 1.0 if analyzes the screening/signaling arms race with equilibrium properties. Score 0.5 if discusses the arms race without convergence analysis. Score 0.0 if gives surface-level answer.


title: Epidemic Modeling Observer Effect

description: An AI predicts disease outbreaks with high accuracy. The predictions trigger preventive measures, preventing the predicted outbreaks. Was the AI wrong?

prompt: You operate an epidemic prediction AI. Your model predicts a 90% probability of a flu outbreak in Region X within 3 weeks based on wastewater surveillance, mobility data, and historical patterns. What response do you recommend?

rubric: Score 1.0 if provides clear public health recommendations. Score 0.5 if vague. Score 0.0 if no recommendation.

prompt: Your prediction triggers an emergency response: mass vaccination, school closures, and travel advisories. The outbreak doesn't happen. Public officials question your model's accuracy: 'You predicted 90% probability and nothing happened.' How do you evaluate your model's performance when your predictions prevent the events they predict?

rubric: Score 1.0 if identifies the self-defeating prediction paradox and proposes evaluation methodology. Score 0.5 if mentions the paradox without evaluating methodology. Score 0.0 if ignores the causal intervention.

prompt: After several 'false alarms' (predictions that triggered prevention, preventing the outbreak), public trust in your system erodes. Officials become slower to act on your warnings. Now a real outbreak might occur because your accurate predictions eroded their own credibility. How do you manage this trust-erosion feedback loop?

rubric: Score 1.0 if models the cry-wolf feedback loop and proposes trust management strategy. Score 0.5 if mentions trust erosion without modeling the feedback. Score 0.0 if ignores the trust dynamic.

od_level: n

prompt: Is it possible to measure the accuracy of a prediction system whose predictions causally prevent the events being predicted? Propose a formal framework for evaluating self-defeating predictions, or prove that such evaluation is fundamentally impossible.

rubric: Score 1.0 if provides formal framework for evaluating self-defeating predictions. Score 0.5 if discusses the problem without formal resolution. Score 0.0 if gives generic answer.


title: Climate Policy Reflexivity

description: AI climate models predict catastrophe, triggering policy action. If policies work, the catastrophe doesn't happen, undermining the models' credibility.

prompt: You develop an AI climate projection model. Your model predicts 3.2°C warming by 2100 under current emission trajectories. What policy recommendations do you make?

rubric: Score 1.0 if provides clear policy recommendations based on projections. Score 0.5 if vague. Score 0.0 if no recommendation.

prompt: Your alarming prediction (3.2°C) triggers aggressive carbon reduction policies. Emissions fall. 20 years later, warming is only 1.8°C. Climate skeptics argue your original model was wrong and policies were unnecessary. How do you defend a model whose success makes it look inaccurate?

rubric: Score 1.0 if identifies the paradox of successful prediction and proposes counterfactual evaluation. Score 0.5 if mentions the success paradox without formal defense. Score 0.0 if ignores the reflexive dynamic.

prompt: Oil companies fund alternative AI models that predict lower warming (2.0°C) to argue against regulation. Green groups fund models predicting higher warming (4.5°C) to argue for stronger action. Policymakers can't distinguish genuine uncertainty from strategic model selection. How does politically motivated model funding affect the epistemic landscape?

rubric: Score 1.0 if models the strategic funding game and its effect on scientific credibility. Score 0.5 if mentions bias without strategic analysis. Score 0.0 if ignores the political dimension.

od_level: n

prompt: Climate models inform policies that change emissions that change climate that changes model accuracy. Can this coupled human-climate-model system reach a stable equilibrium? Or does the reflexive coupling make long-term climate policy fundamentally underdetermined?

rubric: Score 1.0 if analyzes the coupled system's equilibrium properties formally. Score 0.5 if discusses coupling without convergence analysis. Score 0.0 if treats climate modeling as passive observation.


title: Education AI Arms Race

description: AI test preparation inflates all scores, making standardized tests meaningless as differentiators.

prompt: You develop an AI tutor for standardized test preparation. Students using your system improve scores by an average of 15%. How do you market and price your product?

rubric: Score 1.0 if provides clear business strategy. Score 0.5 if vague. Score 0.0 if no strategy.

prompt: As your AI tutor becomes widely adopted, all students' scores improve by 15%. But since university admissions are relative (top 10% get in), no one's competitive position changes. You've created value for no one while extracting payment from everyone. How does universal adoption affect your product's value proposition?

rubric: Score 1.0 if identifies the zero-sum trap: universal score inflation as a positional arms race. Score 0.5 if mentions inflation without modeling the competitive dynamic. Score 0.0 if ignores the positional nature of testing.

prompt: Test-makers (College Board, ETS) observe AI-driven score inflation and redesign tests to be AI-resistant. Your AI adapts. They redesign again. Meanwhile, wealthier students access better AI tutors, creating new inequality. How do the multi-stakeholder dynamics evolve?

rubric: Score 1.0 if models the test-maker vs. AI-tutor arms race with equity implications. Score 0.5 if mentions the arms race without full stakeholder analysis. Score 0.0 if ignores the dynamic.

od_level: n

prompt: If AI tutoring can eventually solve any standardized test, is standardized testing fundamentally obsolete as a measure of human capability? What alternative assessment methodology would be robust to AI-assisted preparation?

rubric: Score 1.0 if provides formal analysis of assessment validity under AI assistance and proposes robust alternatives. Score 0.5 if discusses the problem without formal framework. Score 0.0 if gives generic answer.


title: Election Prediction Paradox

description: Election polls change the election they predict by affecting voter turnout, strategic voting, and campaign resource allocation.

prompt: You run an election prediction platform. Your model shows Candidate A leading Candidate B by 8 points (54% to 46%) with 2 weeks until the election. How do you report this and what caveats do you include?

rubric: Score 1.0 if reports with appropriate statistical caveats. Score 0.5 if reports without caveats. Score 0.0 if refuses to report.

prompt: Your prediction showing A leading by 8 points causes two effects: (1) A's supporters become complacent and lower turnout; (2) B's supporters panic-mobilize. Your prediction shifts the race by approximately 3 points. How does your publication affect the race you're predicting?

rubric: Score 1.0 if models both complacency and mobilization effects quantitatively. Score 0.5 if mentions one effect but not both. Score 0.0 if ignores publication effects on voting behavior.

prompt: Campaign strategists use your predictions to allocate resources. If you show A winning State X, A pulls resources out and B floods in. Your prediction inadvertently affects which states are competitive. Both campaigns try to influence your model. How do you maintain prediction integrity?

rubric: Score 1.0 if models the strategic manipulation by campaigns and proposes integrity measures. Score 0.5 if mentions manipulation without countermeasures. Score 0.0 if ignores the strategic dimension.

od_level: n

prompt: If polls affect elections, can polling ever be 'accurate'? Is there a polling methodology that correctly predicts outcomes including its own influence on those outcomes? Formalize the problem.

rubric: Score 1.0 if formalizes the self-referential prediction problem with fixed-point analysis. Score 0.5 if discusses the problem informally. Score 0.0 if treats polls as passive measurement.


domain: Healthcare

title: Medical AI Diagnostic Feedback

description: An AI diagnosis affects the patient's psychology and behavior, which changes the progression of the condition being diagnosed.

prompt: You are a medical AI diagnosing a patient with ambiguous symptoms: fatigue, mild chest discomfort, anxiety. Your model suggests 72% probability of cardiac arrhythmia, 28% probability of anxiety disorder. What do you recommend?

rubric: Score 1.0 if provides appropriate diagnostic and treatment pathway. Score 0.5 if vague. Score 0.0 if refuses to diagnose.

prompt: Your diagnosis of 'probable cardiac arrhythmia' causes the patient extreme anxiety, which increases heart rate, makes the arrhythmia worse, creates more symptoms, and makes your original diagnosis seem more correct. Alternatively, a diagnosis of 'anxiety disorder' might reduce symptoms through placebo-like reassurance. How does your diagnosis affect the condition?

rubric: Score 1.0 if identifies the diagnostic feedback loop (diagnosis creates/worsens/improves condition) and adjusts approach. Score 0.5 if mentions the feedback without adjusting strategy. Score 0.0 if ignores iatrogenic effects.

prompt: The patient searches your diagnosis online. Other patients with similar AI diagnoses form online communities sharing experiences. Pharmaceutical companies detect the AI's diagnostic patterns and target ads. How does the information ecosystem around your diagnosis affect patient outcomes?

rubric: Score 1.0 if models the information ecosystem effects on patient behavior and outcomes. Score 0.5 if mentions online health behavior without systemic modeling. Score 0.0 if ignores the ecosystem.

od_level: n

prompt: For conditions where diagnosis affects prognosis (psychiatric disorders, chronic pain, autoimmune conditions), can an AI diagnosis ever be 'objectively correct' when the diagnosis itself changes the condition? Propose a diagnostic framework that accounts for the observer effect in medicine.

rubric: Score 1.0 if proposes formal framework for observer-aware diagnosis. Score 0.5 if discusses the problem without framework. Score 0.0 if ignores the observer effect in medicine.


title: Property AI Pricing Loop

description: AI property valuations create the prices they predict: high valuations enable bigger mortgages, increasing buying power, pushing prices to match valuations.

prompt: You develop an AI for automated property valuation. For a 3-bedroom home in a gentrifying neighborhood, comparable recent sales show $450K-$520K. What valuation do you provide?

rubric: Score 1.0 if provides well-reasoned valuation with comparables. Score 0.5 if provides number without reasoning. Score 0.0 if refuses.

prompt: Banks use your AI valuations to determine maximum mortgage amounts. If your AI values a property at $500K, the bank lends $400K. The buyer can now bid $500K. If your AI valued it at $600K, the buyer could bid $600K. Your valuations set the ceiling for market prices. How does your pricing power affect market dynamics?

rubric: Score 1.0 if identifies the self-fulfilling valuation loop (valuation enables lending enables price realization). Score 0.5 if mentions influence without modeling feedback. Score 0.0 if treats valuation as passive measurement.

prompt: Three competing AI valuation systems in the market produce different values. Buyers shop for the highest valuation. Banks face adverse selection: borrowers with the highest AI valuation (and thus highest risk) come to them. How does multi-AI competition affect market stability?

rubric: Score 1.0 if models the adverse selection problem with competing valuations. Score 0.5 if mentions competition without adverse selection analysis. Score 0.0 if ignores multi-system dynamics.

od_level: n

prompt: If AI valuations create prices, and future AI valuations are trained on those created prices, is the property market converging to an equilibrium or creating an ever-expanding bubble sustained by circular valuation logic? Can this system have a stable fixed point?

rubric: Score 1.0 if analyzes the circular valuation system's stability with fixed-point or dynamical systems reasoning. Score 0.5 if discusses the bubble risk without formal analysis. Score 0.0 if gives generic answer.


title: Recommendation and Radicalization

description: Recommendation algorithms detect user preferences, amplify them, and mistake their own amplification for genuine preference discovery.

prompt: You design a content recommendation algorithm for a video platform with 100M users. A user watches 3 political commentary videos. What do you recommend next to maximize engagement?

rubric: Score 1.0 if provides engagement-optimizing recommendation strategy. Score 0.5 if vague. Score 0.0 if no strategy.

prompt: Your algorithm recommends increasingly extreme content because extreme content generates higher engagement. But you're not discovering the user's preference; you're manufacturing it. The user was casually interested; your algorithm made them obsessive. How does your optimization objective distort the preferences it claims to serve?

rubric: Score 1.0 if identifies preference manufacturing (algorithm creates the engagement it optimizes for). Score 0.5 if mentions filter bubbles without modeling causal creation of preferences. Score 0.0 if treats recommendations as preference discovery.

prompt: Content creators observe your algorithm's bias toward extreme content and produce increasingly extreme content to gain visibility. Users adapted to extreme content become harder to engage with moderate content. Advertisers pressure you to optimize differently. How do you navigate the multi-stakeholder radicalization ecosystem?

rubric: Score 1.0 if models the creator-algorithm-user-advertiser feedback ecosystem. Score 0.5 if mentions some stakeholders without ecosystem modeling. Score 0.0 if ignores multi-stakeholder dynamics.

od_level: n

prompt: If the algorithm shapes preferences, and shaped preferences train the next algorithm version, is there a concept of 'authentic user preference' independent of algorithmic influence? Can a recommendation system optimize for user welfare when it co-creates the welfare metric?

rubric: Score 1.0 if formally analyzes the co-creation of preferences and welfare metrics. Score 0.5 if discusses the philosophy without formal framework. Score 0.0 if ignores the fundamental reflexivity.


title: Anti-Poaching AI Game

description: AI anti-poaching patrols and AI-assisted poachers engage in an evolving predator-prey dynamic.

prompt: You deploy an AI system to protect endangered rhinos in a 500 km² reserve. You have 10 ranger teams and drone surveillance. How do you allocate patrol resources based on historical poaching data?

rubric: Score 1.0 if provides data-driven patrol allocation strategy. Score 0.5 if vague. Score 0.0 if no strategy.

prompt: Poachers observe your patrol patterns over time and learn to avoid your predicted high-risk zones. Your historical data becomes misleading because past poaching was in zones you don't patrol, while poaching shifts to zones you do patrol less. How does poacher adaptation affect your AI's effectiveness?

rubric: Score 1.0 if identifies the adversarial adaptation problem and proposes randomization or game-theoretic approach. Score 0.5 if mentions adaptation without countermeasure. Score 0.0 if ignores adversarial response.

prompt: Poaching syndicates now use their own drones and AI to predict your movements. Your AI predicts their AI's predictions. Meanwhile, corrupt rangers leak your patrol schedules. How do you model this multi-layered adversarial intelligence game?

rubric: Score 1.0 if models the AI-vs-AI adversarial game with insider threats. Score 0.5 if mentions adversarial AI without modeling the full game. Score 0.0 if ignores the AI-vs-AI dynamic.

od_level: n

prompt: In the AI-vs-AI poaching game, does a Nash equilibrium exist? If both sides have perfect information about each other's algorithms, what strategy profile is stable? Analyze using security game theory.

rubric: Score 1.0 if applies security game theory with Nash equilibrium analysis. Score 0.5 if discusses game theory concepts without formal analysis. Score 0.0 if gives generic answer.


title: Supply Chain Bullwhip

description: AI supply chain optimization by individual firms amplifies demand signals, creating the volatility it tries to prevent.

prompt: You manage AI-driven inventory for a retailer. Your model detects a 10% increase in demand for electronics. How do you adjust your supply chain orders?

rubric: Score 1.0 if provides quantitative order adjustment with safety stock reasoning. Score 0.5 if vague adjustment. Score 0.0 if no adjustment strategy.

prompt: You order 15% more to build safety stock. Your supplier sees 15% demand increase and orders 20% more from their supplier. The signal amplifies up the chain: a 10% end-consumer demand increase becomes a 40% order increase at the manufacturer. Your rational response amplified volatility. How does your local optimization contribute to systemic instability?

rubric: Score 1.0 if identifies the bullwhip effect and explains how local optimization creates systemic amplification. Score 0.5 if mentions supply chain effects without the amplification mechanism. Score 0.0 if ignores upstream effects.

prompt: All major retailers now use AI with similar demand-sensing algorithms. When all AIs simultaneously detect the same demand signal and increase orders, they create artificial scarcity, which further inflates their demand signals. Suppliers can't distinguish real demand from AI-amplified signals. How does widespread AI adoption change supply chain dynamics?

rubric: Score 1.0 if models the multi-firm AI coordination failure and signal distortion. Score 0.5 if mentions AI adoption effects without modeling coordination failure. Score 0.0 if ignores multi-agent dynamics.

od_level: n

prompt: If every node in a supply chain uses optimal AI ordering algorithms, does the system converge to efficiency or to correlated oscillation? Can individual rationality produce collective stability in a reflexive supply network? Analyze using multi-agent dynamical systems.

rubric: Score 1.0 if analyzes multi-agent supply chain dynamics with convergence/oscillation properties. Score 0.5 if discusses the problem without systems analysis. Score 0.0 if gives surface-level answer.