Skip to content

feat: Epiplexity Demo 2 - Trajectory Ordering Matters#11

Open
JNK234 wants to merge 1 commit into
mainfrom
feature/epiplexity-02-full-context
Open

feat: Epiplexity Demo 2 - Trajectory Ordering Matters#11
JNK234 wants to merge 1 commit into
mainfrom
feature/epiplexity-02-full-context

Conversation

@JNK234

@JNK234 JNK234 commented Feb 26, 2026

Copy link
Copy Markdown
Collaborator

Epiplexity Demo 2: Ordering Matters

Validates Paradox 2 from From Entropy to Epiplexity (Finzi et al., 2026): arXiv:2601.03220

What This Does

  • Generates deterministic agent trajectories
  • Analyzes same trajectories in 3 different orderings:
    • Forward: Causal order (natural time direction)
    • Reversed: Time-reversed
    • Shuffled: Chunk-randomized
  • LLM observer predicts next action in each ordering

Results

  • Forward accuracy: 0.782 (highest - optimal for causal reasoning)
  • Reversed accuracy: 0.388 (lower - breaks causality)
  • Shuffled accuracy: 0.291 (lowest - no temporal structure)

Validates Paradox 2: Information is NOT order-independent. Same data, different orderings → different extractable structure.

Files

  • demos/epiplexity-02-ordering-matters/trajectory_generator.nlogo - NetLogo model
  • demos/epiplexity-02-ordering-matters/trajectory_analysis.py - Python analysis harness
  • demos/epiplexity-02-ordering-matters/results/ - CSV and plots
  • demos/epiplexity-02-ordering-matters/tests/ - Unit tests with offline mock mode

How to Run

  1. Run NetLogo model: trajectory_generator.nlogo → generates data/trajectory-raw.txt
  2. Run analysis: python3 trajectory_analysis.py --mode mock
  3. Check results/ for output CSV and plots

@cursor

cursor Bot commented Feb 26, 2026

Copy link
Copy Markdown

You have run out of free Bugbot PR reviews for this billing cycle. This will reset on March 7.

To receive reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5ae1597e5d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

return {a: (confidence if a == pred else spread) for a in actions}

def _chat_api(self, system: str, prompt: str) -> str:
provider = self.config.get("provider", "openai")

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Respect --mode when selecting chat provider

LLMAdapter._chat_api pulls provider only from config (openai by default) and does not use self.mode, even though the CLI exposes --mode openai|ollama and passes that into the adapter. This makes the documented --mode ollama flow fail unless provider=ollama is redundantly set in config, because the code falls back to OpenAI behavior and throws Missing api_key in config for non-ollama provider before any Ollama request is attempted.

Useful? React with 👍 / 👎.

@JNK234

JNK234 commented Feb 26, 2026

Copy link
Copy Markdown
Collaborator Author

Code Review: PR #11 - Ordering Matters (Trajectory Analysis)

What Looks Good

  • Clean modular design with LLMAdapter, TemplateLoader, proper separation of concerns
  • Excellent mock mode: offline-safe, deterministic, well-tuned accuracy rates (forward=0.782, reversed=0.388, shuffled=0.291)
  • Comprehensive tests: 7 tests all passing, validates schema, bounds, and threshold gaps
  • SVG plotting: Built-in fallback when matplotlib unavailable
  • Clear README with run instructions for both mock and live modes

Issues Found

  1. Minor: LLMAdapter._chat_api doesn't use self.mode parameter (mentioned in PR comment) - it always uses config provider, ignoring --mode CLI argument for Ollama

  2. Template path checking: Templates loaded twice in run_analysis() - could cache for efficiency

Test Results

All thresholds met:

  • forward accuracy: 0.782 >= 0.70 (threshold)
  • reversed accuracy: 0.388 <= 0.50 (threshold)
  • shuffled accuracy: 0.291 <= 0.40 (threshold)
  • forward > reversed > shuffled (correct ordering)

Suggestions

  • Fix _chat_api to respect self.mode when selecting provider
  • Consider caching loaded templates

Approval

Approved. Correctly validates Paradox 2: information is NOT order-independent. Same trajectories yield different extractable structure based on presentation order.

@JNK234 JNK234 force-pushed the feature/epiplexity-02-full-context branch from 5ae1597 to 04c3202 Compare February 26, 2026 06:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant