[Deep Dive] From Karpathy’s Second Brain to Entropy: A Practical Architecture for AI-First Work

Jay Khalife took the idea behind Andrej Karpathy’s LLM-maintained wiki and extended it beyond personal knowledge management into an operational system for customer strategy, simulation, and action

A lot of people have seen Andrej Karpathy’s second-brain idea by now.

The basic pattern is elegant: instead of relying on an LLM to rediscover what matters from a pile of raw notes every time you ask a question, you let the model help maintain a structured, interlinked wiki of markdown files. The LLM is not just a chat interface sitting on top of your knowledge. It becomes the mechanism that organizes, summarizes, links, and continuously updates that knowledge over time.

That shift matters.

It changes the role of AI from “search and summarize” to something closer to “maintain a navigable working memory.”

What made Jay Khalife’s Entropy system interesting is that he took that pattern in a much more operational direction.

He didn’t stop at building a second brain for himself.

He built a system that applies similar ideas to customers, portfolios, product knowledge, scenario generation, and tailored playbooks. In other words, he pushed the second-brain pattern out of personal knowledge management and into live decision support.

That is what makes Entropy worth paying attention to.

Karpathy’s core idea: pre-compiled knowledge beats repeated rediscovery

At the heart of the Karpathy-style approach is a simple observation: many AI workflows waste effort by repeatedly re-reading large volumes of unstructured source material.

The alternative is to maintain a layered system:

• raw sources as immutable inputs

• an LLM-maintained wiki of structured summaries, entities, concepts, and syntheses

• a schema or contract that governs how the AI maintains and queries that knowledge base

At a practical level, the pattern is easiest to understand as a three-layer architecture:

1. raw/ for source materials

2. wiki/ for LLM-compiled knowledge pages

3. CLAUDE.md for schema, conventions, and operating instructions

The key insight is not that markdown is magical.

The key insight is that compiled, linked, human-readable knowledge is easier for both people and models to navigate than a shapeless pile of documents.

That gives you several benefits:

• knowledge becomes portable across tools

• concepts can be updated rather than duplicated

• the graph gets denser over time

• synthesis becomes easier because relationships are already partially expressed in the structure

• the model spends less effort rehydrating context from scratch

That is the seed idea.

Where Jay extended the pattern

Jay’s work takes that seed idea and pushes it in four important directions.

1. From personal knowledge to operator-specific intelligence

Karpathy’s pattern starts with a personal knowledge base.

Jay kept that idea, but made it explicitly role-aware. His second brain was not just a pile of notes. It was organized around his working context as a portfolio leader: sales frameworks, decision-making patterns, concepts, mental models, and personal tendencies relevant to how he operates.

That matters because an operational system should not only know the domain. It should know the operator.

In Jay’s framing, the system should be able to reflect not just customer context, but the actual human being responsible for the account. That makes the resulting recommendations more specific than generic playbooks ever could be.

2. From one brain to many customer-specific intelligence layers

This is the first big leap.

Instead of stopping at a personal wiki, Jay created intelligence summaries or second-brain-like structures for individual customers as well.

In his presentation, those customer layers pulled from multiple systems:

• email threads

• meeting transcripts

• Jira tickets

• CRM context

• support history

• stakeholder information

• account-level signals and friction points

This changes the knowledge graph from “what I know” into “what I know about this customer, this relationship, and this situation.”

That is a major upgrade in usefulness.

A normal second brain helps you think. A customer intelligence layer helps you act.

3. From customer layers to a portfolio brain

Jay then connects those customer-level structures into a portfolio-wide view.

That is another important architectural step.

Once accounts are represented in a consistent way, you can start looking across them for patterns:

• common friction points

• shared risk signals

• repeated pricing issues

• recurring stakeholder dynamics

• similarities across segments, geographies, or product lines

This is where a knowledge graph starts behaving less like a notebook and more like an operational map.

Instead of querying isolated documents, you’re navigating a structured representation of the portfolio.

That makes higher-order questions possible:

• Which accounts are most at risk and why?

• Which patterns recur across similar customers?

• Where are the operational bottlenecks?

• What should get attention first this week?

That is a very different kind of AI value than “summarize my notes.”

4. From knowledge organization to scenario simulation and playbooks

This is the second big leap.

Entropy does not stop at retrieval or structured navigation. It uses the knowledge layers as the basis for generating strategic scenarios and playbooks.

That means the system is not merely answering factual queries. It is helping explore possible moves.

In Jay’s presentation, that included things like:

• different sales frameworks

• pricing strategies

• outreach approaches

• timing differences

• channel choices

• coaching notes for Jay himself

• stakeholder-aware recommendations

The output is not a generic best-practice memo. It is a contextual playbook tailored to both the customer and the operator.

This is where the second-brain pattern becomes a decision-support architecture.

Why this is more than “better RAG” language

Jay contrasts his approach with traditional RAG. That instinct makes sense, but it is worth framing carefully.

The strongest technical claim is not that this architecture somehow eliminates retrieval problems forever.

The stronger claim is this:

A graph-structured, LLM-maintained knowledge base reduces the need to repeatedly reconstruct context from raw documents, and makes operational reasoning more targeted, portable, and cumulative.

That is a big improvement even without making sweeping claims.

In practice, this kind of system can help by:

• preserving conceptual structure between sessions

• turning repeated source material into reusable pages

• making relationships explicit through links and hubs

• separating raw evidence from synthesized knowledge

• allowing the model to navigate curated abstractions rather than starting from raw text every time

That is a meaningful architectural difference.

The feedback loop is what turns it into a system

The most important addition Jay described may actually be the simplest one: the feedback loop.

The system proposes a likely path. Jay executes in the real world. The observed outcome gets fed back into the process.

That matters because it prevents the system from becoming a static knowledge artifact.

Without feedback, you have a smart reference layer. With feedback, you have the beginnings of a learning operational system.

Even if the loop is partially manual today, the architecture matters. It creates a place where predicted outcomes can be compared against observed reality.

That is the difference between:

• “here is a plausible answer”

• and “here is a recommendation that can be calibrated over time”

In operational work, that difference is everything.

Another useful detail: Entropy seems designed for portability

One subtle but important theme in Jay’s presentation is portability.

Karpathy’s pattern already leans into portability because markdown files, folders, and explicit schemas are durable and tool-agnostic.

Jay preserves that principle.

Instead of overinvesting in a custom application layer, he used tools that were already strong in their respective jobs:

• Notion for structured operational views

• Obsidian for linked knowledge and graph-like navigation

• Claude for maintenance, synthesis, and execution support

That matters for a practical reason: if the knowledge is trapped in one proprietary interface, it is much harder to evolve.

If the knowledge is represented as files, summaries, relationships, and conventions, the architecture can move.

That is exactly what you want from an AI-native operating model.

The school-side reuse proves the pattern is real

One of the strongest signals in the whole story came after the original presentation.

Someone on the school side took the core architecture and adapted it to a very different operational domain with its own structured operational data, doctrine, and evaluation frameworks.

The follow-on implementation showed that the architecture mapped almost perfectly.

That matters because it shows the pattern is not limited to one commercial role.

Even more interesting, the adaptation surfaced both:

• actionable operational insight right away

• and practical implementation lessons

For example:

• a markdown-only workflow was not ideal for pre-structured JSON data

• a Python script was the better way to generate intelligence summaries and hub nodes at scale

• existing agent scaffolding could be adapted by appending Entropy navigation rather than replacing everything

That is what real architecture reuse looks like.

A pattern hits another domain. The structure holds. The implementation bends where it needs to.

There isn’t one starting point

One nuance matters here: not everyone begins from the same place.

Jay’s presentation seems to assume a culture where people already know what WorkFlowy is and many already have some version of a second brain. That may be true in his immediate environment. It is not universally true.

In practice, there are at least three valid entry points into this pattern.

1. Second-brain-first

This is the Jay path.

You already have a body of personal knowledge, notes, frameworks, or a maintained second brain. The job is to convert that into a more structured, navigable, AI-maintained system and then extend it into operational use.

2. Ingestion-first

This is often the more realistic path for people who do not already have a polished second brain.

Instead of starting with a clean personal wiki, you start with a mess: local files, archived notes, meeting artifacts, documents, exports, and scattered source material. The system begins by harvesting, scoring, filtering, and ingesting that material into a more structured knowledge layer over time.

This is a very practical route because most people do not have a beautifully maintained personal knowledge graph sitting around waiting to be used.

3. Structured-data-first

This is the path surfaced by the follow-on adaptation.

If the source material already exists in structured form, JSON, CSV, database exports, SaaS records, or operational system outputs, manual note creation may be the wrong first move. In that case, the better approach is often to generate intelligence summaries, hub nodes, and relationship layers programmatically.

These are not competing philosophies. They are different front doors into the same architecture.

What matters is not whether you started from WorkFlowy, an archive folder, or a JSON export. What matters is whether you move toward a durable system that separates raw material, compiled knowledge, and operational use.

What to do if you don’t already have a second brain

One of the risks in conversations like this is assuming everyone already has a polished note system, a clean WorkFlowy export, or a well-maintained Obsidian graph.

Most people don’t.

They have a mess: files in Downloads, half-forgotten PDFs, meeting notes buried in folders, old markdown, random text files, and project artifacts scattered across machines and cloud storage.

That is where an ingestion-first workflow becomes useful.

A simple version looks like this:

1. Set up a target inbox
   Create a dedicated place where potentially useful material can land for review instead of dumping everything straight into your knowledge graph.

2. Choose the directories that matter
   Start with a small number of high-signal locations like Documents, Downloads, archived notes, and project folders.

3. Deduplicate before you think
   Use content hashing so the same file does not get processed repeatedly just because it was moved, renamed, or copied.

4. Extract a lightweight preview and score it with AI
   Pull a small text preview from each file and use a fast model to decide whether it is worth importing, why, and how it should be categorized.

5. Run a dry pass, then a real ingest
   Test the pipeline first, then import only the files that pass your relevance threshold into the inbox.

6. Review and integrate at human speed
   Let AI do the filtering, but let a person decide what becomes part of the long-term knowledge system, how it gets categorized, and what it links to.

That is not a replacement for the second-brain-first path. It is the on-ramp for people who are starting with a mess.

For readers who want a concrete reference implementation of that ingestion-first pattern, I’ve published one here:

github.com/dp-pcs/second-brain-ingest

A lightweight blueprint for building something similar

We do not yet have enough detail to publish a literal step-by-step implementation manual for Entropy (comment below if that would be useful)

But we do have enough to describe a practical blueprint.

1. Start with the raw/source/compiled split

Borrow the Karpathy pattern directly:

• raw sources are immutable

• compiled knowledge is maintained by the LLM

• schema/instructions govern how the system behaves

That separation is one of the cleanest ideas in the whole approach.

2. Build a role-aware second brain

Don’t just dump notes into a folder. Shape the system around the operator:

• responsibilities

• frameworks

• recurring concepts

• personal patterns

• decision criteria

The system should know what kind of work you actually do.

3. Create intelligence objects for the entities that matter

In Jay’s case, those entities appear to be customers and accounts. In another domain, they might be campuses, vendors, projects, or business units.

The principle is the same: create structured intelligence summaries around the core units of decision-making.

4. Add a portfolio-level or system-level graph

Once entities are represented consistently, connect them. This is where you start getting cross-case pattern recognition instead of isolated summaries.

5. Layer in domain knowledge

Jay added product knowledge and organizational constraints. Another team added security doctrine and quality bars.

This is crucial.

A useful system does not just know facts about entities. It also knows the rules, frameworks, and operating doctrine that shape valid decisions.

6. Use the AI for scenario generation, not just summarization

The unlock comes when the system starts helping answer:

• what are the possible moves?

• what changes if a variable changes?

• which strategy seems strongest?

• what would a playbook look like for this context?

That is where AI-first work starts to feel materially different.

7. Feed outcomes back in

Even if you do this manually at first, create a deliberate place in the workflow for reality to update the model of the world.

Without that, the system is descriptive. With it, the system becomes adaptive.

8. Automate generation when the source data is already structured

One implementation lesson here is straightforward.

If you already have structured JSON or database-style exports, do not romanticize manual note creation. Generate the pages, summaries, and hub nodes programmatically where appropriate.

That is not a departure from the pattern. It is a practical extension of it.

The broader lesson

Karpathy’s second-brain pattern is powerful because it gives LLMs a better substrate for working with knowledge.

Jay’s Entropy system is interesting because it shows what happens when that substrate gets attached to an operational loop.

That is the important move.

Not just “build a wiki.”
Not just “use Obsidian.”
Not just “have AI summarize your notes.”

Build a knowledge architecture that:

• reflects how work is actually done

• represents the entities and decisions that matter

• supports navigation across context

• generates actionable outputs

• learns from what happens next

That is how a second brain starts becoming an operational system.

And that is why Entropy is worth studying, even before every implementation detail is formalized.

A note on scope

One final thing is worth saying plainly: this article describes a pattern and architecture, not a benchmarked reference implementation.

We know enough from the presentation and follow-up examples to explain what is novel and useful here. We do not yet know every prompt, schema, file convention, or scoring mechanism under the hood.

That’s fine.

The value of this write-up is not pretending we have a turnkey open-source manual. The value is showing how an already compelling idea, Karpathy’s LLM-maintained knowledge graph, can be extended into a practical model for AI-first operational work.

That alone is worth attention.

---

Related Links

From Portfolio Management to Predictive Playbooks: How Jay Khalife Built Entropy

---

David Proctor is VP of AI at Trilogy. He writes about AI infrastructure, enterprise automation, and what actually works in production.