XRPL Data Architecture - How the Ledger Stores Information

The XRP Ledger is a distributed database that maintains two types of information:

XRPL DATA HIERARCHY:

LEDGER (snapshot of state at a point in time)
├── Ledger Header
│   ├── Ledger index (sequential number)
│   ├── Close time (when validated)
│   ├── Parent hash (previous ledger)
│   ├── Transaction hash (merkle root of transactions)
│   └── State hash (merkle root of all objects)
│
├── Transaction Set (what changed in this ledger)
│   ├── Transaction 1
│   ├── Transaction 2
│   └── ... (all transactions in this ledger)
│
└── State Tree (all objects after changes)
    ├── AccountRoot objects
    ├── Offer objects
    ├── RippleState (trust line) objects
    ├── AMM objects
    ├── Escrow objects
    └── ... (all object types)

Every 3-5 seconds, a new ledger version is created:

LEDGER CLOSE PROCESS:

Ledger N (validated state)
    │
    ▼
Pending transactions submitted
    │
    ▼
Consensus reached on transaction set
    │
    ▼
Transactions applied to state
    │
    ▼
Ledger N+1 (new validated state)

- Average ledger close: 3-5 seconds
- ~17,000-25,000 ledgers per day
- Each ledger records all transactions AND resulting state

Why this matters for metrics:

• Transaction counts are straightforward: count transactions across ledgers
• "Point in time" analysis is easy: every ledger is a complete snapshot
• Historical analysis requires access to old ledgers (full history nodes)
• Real-time monitoring watches each new ledger as it closes

Each ledger has a unique sequential index number:

LEDGER IDENTIFICATION:

By Index: ledger_index = 85,000,000
By Hash: ledger_hash = "ABC123..." (64 character hex)
By Shortcut:
  - "validated" = latest validated ledger
  - "closed" = latest closed (may not be validated)
  - "current" = working ledger (not yet closed)

- Genesis ledger: index 32570 (not 0—early history was lost)
- Index increases by 1 per ledger
- Current ledger: ~85,000,000+ (as of late 2024)

Practical application:

1. Get current ledger index
2. Calculate approximately how many ledgers back = 24 hours
3. Query all ledgers in that range
4. Aggregate the data

ROUGH CALCULATION:

24 hours = 86,400 seconds
Average ledger time = 4 seconds
Ledgers per day ≈ 86,400 / 4 = 21,600

Last 24 hours ≈ Current index - 21,600

---

XRPL supports specific transaction types, each enabling different activities:

TRANSACTION TYPE CATEGORIES:

PAYMENTS & TRANSFERS
├── Payment: Move XRP or issued currencies
└── PaymentChannelCreate/Claim/Fund: Payment channel operations

DEX OPERATIONS
├── OfferCreate: Place order on DEX
├── OfferCancel: Cancel existing order
└── (Offers auto-cancel when filled)

TRUST & TOKENS
├── TrustSet: Establish/modify trust lines
├── SetRegularKey: Set alternate signing key
└── AccountSet: Modify account settings

AMM OPERATIONS
├── AMMCreate: Create AMM pool
├── AMMDeposit: Add liquidity
├── AMMWithdraw: Remove liquidity
├── AMMVote: Vote on trading fee
└── AMMBid: Bid for auction slot

ESCROW
├── EscrowCreate: Create time/condition locked XRP
├── EscrowFinish: Release escrowed XRP
└── EscrowCancel: Cancel and return escrowed XRP

NFTs
├── NFTokenMint: Create NFT
├── NFTokenBurn: Destroy NFT
├── NFTokenCreateOffer: List NFT for sale
├── NFTokenCancelOffer: Cancel NFT listing
└── NFTokenAcceptOffer: Complete NFT sale

ADVANCED
├── CheckCreate/Cash/Cancel: Check operations
├── DepositPreauth: Authorize deposits
├── Clawback: Issuer reclaims tokens (if enabled)
└── DIDSet/Delete: Decentralized identity

Each transaction type enables specific metrics:

Payment Transactions

PAYMENT ANALYSIS:

What you can extract:
├── Total payment count
├── XRP payment volume
├── Issued currency payment volume
├── Payment destinations (address analysis)
├── Cross-currency payments (pathfinding usage)
├── Partial payment usage
└── Destination tag usage (exchange attribution)

Metric applications:
├── Activity: Daily payment count and volume
├── Adoption: Unique sending/receiving addresses
├── Commercial: Large payment patterns
└── ODL signals: Exchange-to-exchange payment patterns

DEX Transactions (OfferCreate/Cancel)

OFFER ANALYSIS:

What you can extract:
├── Orders placed and canceled
├── Trading pairs activity
├── Order sizes and prices
├── Maker vs taker identification
├── Order book reconstruction
└── Filled order calculations

Metric applications:
├── Liquidity: Order book depth at any historical point
├── Activity: DEX trading volume
├── Market structure: Spread analysis
└── Behavior: Market maker identification

Trust Line Transactions (TrustSet)

TRUSTSET ANALYSIS:

What you can extract:
├── New trust lines established
├── Trust line limits modified
├── Trust lines removed (limit set to 0)
├── Tokens gaining/losing trust
└── Issuer relationship mapping

Metric applications:
├── Ecosystem: Token adoption curves
├── Adoption: Willingness to hold issued assets
├── Risk: Concentration in specific issuers
└── Trends: Which tokens are gaining traction

AMM Transactions

AMM ANALYSIS:

What you can extract:
├── Pool creation and parameters
├── Liquidity additions (who, how much, when)
├── Liquidity withdrawals
├── Trading activity through pools
├── Fee vote patterns
└── Auction slot bidding

Metric applications:
├── Liquidity: TVL tracking over time
├── Yield: LP return calculations
├── Participation: LP address analysis
└── Efficiency: AMM vs order book usage comparison

Every transaction includes metadata with valuable analytical data:

TRANSACTION STRUCTURE:

{
  "TransactionType": "Payment",
  "Account": "rSender...",          // Who initiated
  "Destination": "rReceiver...",    // Who received
  "Amount": "1000000",              // Amount (in drops for XRP)
  "Fee": "12",                      // Fee paid (in drops)
  "Sequence": 47,                   // Account's sequence number
  "SigningPubKey": "...",           // Public key used

"meta": {                         // METADATA - often most valuable
    "TransactionResult": "tesSUCCESS",  // Outcome
    "delivered_amount": "1000000",       // Actual delivered
    "AffectedNodes": [...]               // State changes
  }
}

KEY METADATA FIELDS:

TransactionResult: Did it succeed?
├── "tesSUCCESS" = success
├── "tec..." = claimed fee, failed execution
├── "tef..." = failed, no fee claimed
└── Essential for filtering real activity

delivered_amount: Actual amount received
├── May differ from Amount for partial payments
├── Critical for accurate volume calculations
└── Missing in old transactions (pre-2014)

AffectedNodes: Complete state changes
├── Every object created, modified, deleted
├── Enables balance change reconstruction
└── Most detailed but requires parsing

---

Every XRPL account is represented by an AccountRoot object:

ACCOUNTROOT STRUCTURE:

{
  "LedgerEntryType": "AccountRoot",
  "Account": "rAddress...",
  "Balance": "50000000",            // XRP balance in drops
  "Sequence": 47,                   // Next sequence number
  "OwnerCount": 3,                  // Owned objects count
  "PreviousTxnID": "...",          // Last modifying transaction
  "Flags": 0,                       // Account settings

// Optional fields
  "RegularKey": "...",             // Alternate signing key
  "Domain": "...",                 // Associated domain (hex)
  "EmailHash": "...",              // Gravatar email hash
  "MessageKey": "...",             // Public key for messages
}

ACCOUNT METRICS ENABLED:

Total accounts: Count AccountRoot objects
Account balances: Sum Balance fields
Balance distribution: Analyze Balance distribution
Account age: Compare current ledger to creation transaction
Activity: Track Sequence number changes
```

Trust lines enable holding issued currencies:

RIPPLESTATE STRUCTURE:

{
  "LedgerEntryType": "RippleState",
  "Balance": {
    "value": "100.00",
    "currency": "USD",
    "issuer": "..."                  // Shows direction
  },
  "LowLimit": {
    "value": "1000",
    "currency": "USD", 
    "issuer": "rHolder..."
  },
  "HighLimit": {
    "value": "0",
    "currency": "USD",
    "issuer": "rIssuer..."
  },
  "Flags": ...
}

TRUST LINE METRICS ENABLED:

Total trust lines: Count RippleState objects
By currency: Filter by currency code
By issuer: Filter by issuer address
Token supply: Sum positive balances per issuer
Holder count: Count unique low/high addresses per issuer
Trust growth: Compare counts across ledgers
```

Orders on the decentralized exchange:

OFFER STRUCTURE:

{
  "LedgerEntryType": "Offer",
  "Account": "rTrader...",
  "TakerPays": {                    // What taker gives
    "value": "100.00",
    "currency": "USD",
    "issuer": "..."
  },
  "TakerGets": "10000000",          // What taker receives (XRP drops)
  "Sequence": 47,                    // Unique per account
  "BookDirectory": "...",            // Order book location
  "Flags": ...
}

ORDER BOOK METRICS ENABLED:

Open orders: Count Offer objects
Order book depth: Sum TakerGets at price levels
Bid-ask spreads: Compare best bid/ask prices
Active traders: Count unique Account values
Order distribution: Analyze size distribution
```

Automated Market Maker pools:

AMM OBJECT STRUCTURE:

{
  "LedgerEntryType": "AMM",
  "Asset": {"currency": "XRP"},
  "Asset2": {"currency": "USD", "issuer": "..."},
  "Account": "rAMMAccount...",
  "TradingFee": 500,                // In basis points (0.5%)
  "LPTokenBalance": {...},          // Outstanding LP tokens
  "VoteSlots": [...],               // Fee vote records
  "AuctionSlot": {...}              // Current auction winner
}

AMM METRICS ENABLED:

Total pools: Count AMM objects
TVL: Sum asset values across pools
Pool composition: Track asset ratios
Fee levels: Analyze TradingFee distribution
LP participation: Track LP token distribution
```

Time or condition-locked XRP:

ESCROW STRUCTURE:

{
  "LedgerEntryType": "Escrow",
  "Account": "rCreator...",
  "Destination": "rReceiver...",
  "Amount": "100000000",            // Escrowed XRP
  "FinishAfter": 750000000,         // Time unlock (Ripple time)
  "CancelAfter": 760000000,         // Expiration (Ripple time)
  "Condition": "...",               // Crypto-condition (optional)
  "PreviousTxnID": "..."
}

ESCROW METRICS ENABLED:

Total escrowed: Sum Amount across Escrow objects
Ripple escrow: Filter by known Ripple addresses
Monthly releases: Track EscrowFinish transactions
Return rate: Track EscrowCancel vs EscrowFinish
```

NFTs stored in pages:

NFTOKENPAGE STRUCTURE:

{
  "LedgerEntryType": "NFTokenPage",
  "NFTokens": [
    {
      "NFTokenID": "...",
      "URI": "...",                  // Metadata location
      // Other token properties
    },
    ...
  ],
  "PreviousPageMin": "...",
  "NextPageMin": "..."
}

NFT METRICS ENABLED:

Total NFTs: Count tokens across pages
By issuer: Filter by issuer portion of NFTokenID
Collections: Group by taxon field
Trading activity: Track NFTokenAcceptOffer transactions

---

Not all XRPL nodes store complete history:

NODE TYPES:

FULL HISTORY NODE:
├── Stores every ledger since genesis (~2012)
├── Required for: Historical analysis, complete auditing
├── Cost: Significant storage (multiple TB)
├── Availability: Limited (XRPL Foundation, some validators)

PRUNED/PARTIAL HISTORY NODE:
├── Stores recent ledgers only (configurable retention)
├── Adequate for: Current state queries, recent transactions
├── Cost: Manageable storage (hundreds of GB)
├── Availability: Most public nodes

REPORTING MODE NODE:
├── Optimized for API queries
├── Uses PostgreSQL backend for efficient queries
├── Best for: Analytics, high-volume queries
├── Availability: Some infrastructure providers

Implications for analysis:

QUERY TYPE → NODE REQUIRED:

"Current XRP balance" → Any node
"Transactions in last hour" → Any node
"Transaction history for address" → Full history preferred
"Activity in 2019" → Full history required
"Aggregate metrics since genesis" → Full history + processing

Multiple ways to query XRPL data:

1. WebSocket API (Real-time)

// Direct connection to rippled node
// Best for: Live data, subscriptions

- ledger: Get ledger info
- account_info: Get account state
- account_tx: Get account transactions
- tx: Get transaction details
- book_offers: Get order book
- subscribe: Real-time updates

**2. JSON-RPC API (Request/Response)**

// HTTP-based queries
// Best for: One-off queries, integrations

Same commands as WebSocket
Synchronous request/response model
Easier to integrate with existing systems

**3. Clio Server API (Optimized Queries)**

// Specialized read-only server
// Best for: High-volume analytics queries

• Faster historical lookups
• Optimized for reporting workloads
• Reduced load on validator nodes

4. Data Export Services

// Pre-processed datasets
// Best for: Large-scale analysis, research

- XRPL Foundation data exports
- Third-party analytics providers
- Academic datasets

Example: Getting Daily Transaction Count

// Approach 1: Query each ledger in range
// Precise but slow for large ranges

for (ledger_index = start; ledger_index <= end; ledger_index++) {
  const ledger = await api.ledger({
    ledger_index: ledger_index,
    transactions: true
  });
  transaction_count += ledger.transactions.length;
}

// Approach 2: Use account_tx for specific accounts
// Good for address-specific analysis

const txHistory = await api.account_tx({
  account: "rAddress...",
  ledger_index_min: start,
  ledger_index_max: end
});

// Approach 3: Subscribe to ledger closes
// Good for real-time monitoring

api.subscribe({streams: ["ledger"]});
api.on("ledgerClosed", (ledger) => {
  // Process each ledger as it closes
});

---

Some information simply doesn't exist in XRPL data:

NOT AVAILABLE ON-CHAIN:

IDENTITY INFORMATION
├── Who owns addresses (pseudonymous only)
├── Geographic location of users
├── Legal entity information
├── KYC/AML data
└── This is by design (privacy)

INTENT INFORMATION
├── Why a transaction was made
├── Whether activity is speculative vs commercial
├── Relationship between parties
└── This requires external context

OFF-LEDGER ACTIVITY
├── Central exchange trading (CEX order books)
├── OTC trades
├── Custodial transactions
├── Layer 2 or sidechain activity
└── This happens outside XRPL

ECONOMIC CONTEXT
├── USD value at time of transaction
├── Market conditions
├── Reason for price (news, sentiment)
└── Requires correlation with external data

The 2012-2014 Gap:

• Genesis ledger data partially lost
• Some early transaction metadata incomplete
• delivered_amount field not present before ~2014
• Some early features worked differently

HISTORICAL ANALYSIS CAVEATS:

2012-2013: Limited data quality
2014-2016: Improving but check metadata
2017+: Generally complete and reliable
2020+: Full modern feature set

Reconstruction Requirements:

1. Process every ledger sequentially
2. Track state changes through AffectedNodes
3. Rebuild point-in-time snapshots
4. Handle deprecated features differently

Even perfect on-chain data can't answer:

ATTRIBUTION CHALLENGES:

Q: "How much ODL volume is there?"
A: Can't definitively identify ODL vs arbitrage

Q: "How many unique users?"
A: Can count addresses, not people

Q: "What's the institutional holding?"
A: Can see large balances, can't identify institutions

Q: "Is this account a whale or an exchange?"
A: Sometimes inferable, never certain

MITIGATION:
├── Pattern analysis for probable attribution
├── Cross-reference with disclosed addresses
├── Use ranges rather than precise claims
└── Acknowledge uncertainty explicitly

---

✅ XRPL's data architecture provides complete transaction history since inception

✅ Every ledger object type has a defined structure enabling systematic querying

✅ Transaction metadata contains detailed information about state changes

✅ Multiple API access methods exist for different analytical needs

⚠️ Full history node availability may limit some historical analyses

⚠️ Attribution of addresses to entities remains fundamentally limited

⚠️ Early ledger data (2012-2014) may have gaps or quality issues

⚠️ Optimal querying strategies depend on specific analytical needs

📌 Assuming on-chain data tells the complete story (misses off-chain activity)

📌 Using deprecated or inconsistent historical data without adjustment

📌 Over-engineering data collection when simpler methods suffice

📌 Trusting third-party data without verifying against raw ledger data

XRPL's data architecture is remarkably well-designed for analysis—every transaction is recorded, every state change is traceable, and multiple access methods exist. But architecture doesn't solve the fundamental limitations: addresses aren't identities, activity doesn't reveal intent, and on-chain data captures only what happens on XRPL. Understanding both the capabilities and limitations of this data architecture is essential for honest analysis.

---

Assignment: Create a comprehensive reference document mapping XRPL data structures to the metrics they enable, with example queries for each.

Requirements:

Part 1: Transaction Type Matrix (35%)

Create a detailed matrix covering all major transaction types:

Transaction Type	Fields Available	Metrics Enabled	Example Query	Limitations
Payment	Account, Destination, Amount, delivered_amount, Fee	Volume, count, flow analysis	[Your query]	[What you can't know]
[Continue for all types]

Minimum: 12 transaction types documented

Part 2: Ledger Object Reference (35%)

• Object structure (key fields)
• Metrics extractable from this object type
• How to query (API command)
• Common analysis patterns
• Limitations

Cover minimum: AccountRoot, RippleState, Offer, AMM, Escrow, NFTokenPage

Part 3: Data Access Decision Tree (20%)

Create a flowchart/decision tree for:
"Given analysis need X, what data source and query approach should I use?"

• Current state queries
• Recent history (last 24h-7d)
• Extended history (months-years)
• Real-time monitoring
• Large-scale aggregation

Part 4: Limitation Documentation (10%)

• 10 common analytical questions that can't be definitively answered

• Why each is unanswerable from on-chain data

• Best available approximation approaches

• Completeness of transaction type coverage (20%)

• Accuracy of object structure documentation (20%)

• Practical usefulness of decision tree (20%)

• Quality of example queries (20%)

• Honest limitation acknowledgment (10%)

• Clear organization and presentation (10%)

Time investment: 3-4 hours
Value: This reference document will serve as your personal data dictionary throughout the course and beyond. When you need to extract a specific metric, you'll know exactly where the data lives and how to get it.

---

1. Ledger Structure:

What is the relationship between ledger index, transactions, and ledger objects?

A) Each ledger index contains multiple transactions, which modify ledger objects
B) Each transaction has a unique ledger index and creates one object
C) Ledger objects are created first, then indexed, then transacted
D) Transactions and objects exist independently; ledger index is only for ordering

Correct Answer: A

Explanation: Each ledger (identified by its index) contains a set of transactions that were validated together. These transactions modify the state tree of ledger objects—creating new objects, modifying existing ones, or deleting them. The ledger captures both the transaction history AND the resulting state after those transactions. Answer B confuses the hierarchy (many transactions per ledger). C reverses the causal relationship. D incorrectly separates interrelated concepts.

---

2. Transaction Metadata:

Why is the delivered_amount field in transaction metadata often more important than the Amount field?

A) The Amount field is always in drops, while delivered_amount is in human-readable format
B) Partial payments may deliver less than Amount; delivered_amount shows actual receipt
C) Amount is the transaction fee, delivered_amount is the transfer value
D) delivered_amount accounts for exchange rate changes during settlement

Correct Answer: B

Explanation: The Amount field in a Payment transaction is the intended amount, but if partial payments are enabled, the actual delivered amount may be less. The delivered_amount in metadata shows what the destination actually received. This is critical for accurate volume calculations—using Amount instead of delivered_amount could significantly overstate actual transfer volumes. A is incorrect (both can be in drops). C confuses Amount with Fee. D describes a mechanism that doesn't exist in XRPL (no delayed settlement).

---

3. Object Types:

A researcher wants to analyze the distribution of XRP holdings across accounts. Which ledger object type contains the necessary information?

A) RippleState, because it tracks all balances including XRP
B) AccountRoot, because it contains the Balance field for native XRP
C) Offer, because it shows how much XRP accounts are willing to trade
D) LedgerHashes, because it provides account balance history

Correct Answer: B

Explanation: AccountRoot objects contain the native XRP balance for each account in the Balance field. To analyze XRP distribution, you'd query all AccountRoot objects and analyze their Balance values. RippleState (A) tracks trust line balances for issued currencies, not native XRP. Offer objects (C) show trading intentions, not holdings. LedgerHashes (D) is not an object type that stores account balances—this is fabricated.

---

4. Data Access:

An analyst needs to calculate total DEX trading volume for all of 2023. What is the primary technical challenge?

A) DEX volume data is not recorded on the XRP Ledger
B) Requires full-history node access to query historical ledgers
C) OfferCreate transactions don't include price information
D) The XRP Ledger only stores 30 days of transaction history

Correct Answer: B

Explanation: To calculate 2023 trading volume, you need to query every relevant transaction across all of 2023's ~7+ million ledgers. This requires a full-history node because pruned nodes only retain recent data. Once you have access, the calculation is straightforward—DEX activity IS recorded (A is wrong). OfferCreate transactions DO include price via TakerGets/TakerPays (C is wrong). XRPL stores complete history permanently, not just 30 days (D is wrong)—the limitation is node storage configuration, not protocol design.

---

5. Limitations:

Which of the following questions CANNOT be definitively answered using only on-chain XRPL data?

A) How many XRP were transferred between two specific addresses on a given date?
B) What was the total number of successful Payment transactions last month?
C) What percentage of large transactions represent institutional investors?
D) What is the current balance of any specific XRPL address?

Correct Answer: C

Explanation: Questions A, B, and D are all directly answerable from on-chain data: A through transaction records, B through filtering and counting transactions, D through querying AccountRoot objects. Question C asks about who is behind the transactions (institutional vs retail)—this is the attribution problem. On-chain data shows what happened (large transactions) but not who did it (institutions vs wealthy individuals vs exchanges). There's no on-chain field identifying "institutional investor."

---

For Next Lesson:
Lesson 3 covers the practical data sources and tools you'll use—block explorers, analytics platforms, APIs, and third-party services. With the architectural understanding from this lesson, you'll be able to evaluate which tools actually provide reliable data.

---

End of Lesson 2

Total words: ~6,100
Estimated completion time: 50 minutes reading + 3-4 hours for deliverable