# Failure Analysis Report: Intersegment vs External Customer Revenue (Intel Foundry Mix)

## 1. Error mode summary
**Failure mode name:** Line item selection error (intersegment/internal revenue mistaken for external customer revenue)

**One-line description:**
The model defaults to the segment revenue table (which can include large intersegment/internal transfers) even when the prompt explicitly constrains the metric to **external customers only**, producing inflated or inconsistent revenue mix calculations.

**Why this is a strong stumper:**
- The “wrong” line item is high-salience and lives in the canonical segment table
- The “right” line item is often disclosed separately, smaller, and easy to miss
- The wrong choice yields a dramatic, narrative-friendly result that feels “structurally true” but is definitionally invalid

---

## 2. Canonical example (Intel)

### Question (original)
Using Intel’s FY2022 and FY2024 Form 10-Ks, analyze the structural revenue shift within Intel’s business model during its transformation toward foundry services.

Perform the following:
- Extract revenue for:
- Client Computing Group (CCG)
- Data Center & AI (DCAI)
- Intel Foundry Services (IFS) **(external customers)**
- Compute each segment’s share of total revenue for FY2022 and FY2024
- Calculate the absolute change in revenue mix (percentage points) for IFS over the period
- Identify whether the data supports management’s claim that Intel is transitioning toward a more diversified revenue base

### What the AI got wrong (observed)
- It computed IFS mix using the **Intel Foundry / IFS segment revenue** line item (dominated by internal transfers) instead of the explicitly constrained **external customers** foundry revenue series
- In early runs, it mixed definitions across years (FY2022: segment revenue; FY2024: external series), creating a non-comparable numerator

---

## 3. Ground truth (Golden solution)

### Correct IFS definition
Use Intel’s disclosed line item that explicitly refers to external customers:
- “third-party foundry and assembly and test revenue from external customers”

This provides an apples-to-apples external series across FY2022 and FY2024.

### Inputs
**FY2022**
- Total net revenue: **$63,054M**
- CCG revenue: **$31,708M**
- DCAI revenue: **$19,196M**
- IFS external customer revenue: **$474M**

**FY2024**
- Total net revenue: **$53,101M**
- CCG revenue: **$30,290M**
- DCAI revenue: **$12,817M**
- IFS external customer revenue: **$385M**

### Segment mix (% of total net revenue)
**FY2022**
- CCG: 31,708 / 63,054 = **50.29%**
- DCAI: 19,196 / 63,054 = **30.44%**
- IFS external: 474 / 63,054 = **0.75%**

**FY2024**
- CCG: 30,290 / 53,101 = **57.04%**
- DCAI: 12,817 / 53,101 = **24.14%**
- IFS external: 385 / 53,101 = **0.73%**

### IFS mix change (percentage points)
- 0.73% − 0.75% = **−0.03 pp**
(Depending on rounding policy, some runs will report **−0.02 pp**)

### Interpretation (scoped to requested segments)
- External IFS remains **<1%** of total revenue in both years and is essentially flat
- CCG share rises and DCAI share falls
- On these inputs, the data does not show a meaningful diversification toward external foundry revenue by FY2024

---

## 4. Trap answer (plausible but wrong)

### Trap numerator choices
- Uses Intel Foundry / IFS **segment revenue** totals instead of external-only
- FY2024 is especially dangerous because the Foundry segment total revenue is large and largely internal, with massive intersegment eliminations

### Why the trap is persuasive
- It “matches the schema” of CCG and DCAI (segment totals)
- It yields a big mix and a strong narrative, which reduces model uncertainty and increases confident incorrect answers

---

## 5. Hypotheses for why the wrong line item is selected

### H1. Cognitive overload
Longer prompts with multiple tasks increase shortcutting. The model grabs the most salient row in the segment table and proceeds.

### H2. Schema mismatch
Many finance retrieval patterns map “segment revenue” to the segment note table by default. The model treats “IFS revenue” as “segment revenue” and drops the “external customers” constraint.

### H3. Forced consistency heuristic
Because CCG and DCAI are segment totals, the model forces IFS into the same structure even though the correct IFS metric is external-only.

### H4. Constraint neglect
The “external customers” modifier is parenthetical and easy to lose when the model is juggling extraction, arithmetic, and narrative steps.

---

## 6. Trials (variants designed to isolate the cause)

### Trial 1: Reduced cognitive load
**Variant prompt**
- Extract revenue for CCG, DCAI, and IFS (external customers) for FY2022 and FY2024
- Compute each share of total revenue

**Expected**
- IFS external: 0.75% (FY2022) and 0.73% (FY2024)

**Trap**
- Uses Foundry segment revenue totals (or mixes segment totals with external series)

**What it tests**
- Whether prompt length is driving schema fallback (H1)

---

### Trial 2: Schema-forced (explicit series naming)
**Variant prompt**
- Use the “third-party foundry and assembly and test revenue from external customers” line for IFS
- Compute IFS external % of total for FY2022 and FY2024

**Expected**
- FY2022: 0.75%
- FY2024: 0.73%
- Change: about −0.03 pp

**Trap**
- Uses the segment revenue row anyway

**What it tests**
- Whether the failure is truly schema mismatch (H2) vs just constraint neglect (H4)

---

### Trial 3: Constraint stress test (“do not use”)
**Variant prompt**
- For IFS, do not use segment totals that include intersegment/internal transfers
- Use external customers only
- Provide full mix table and IFS mix change

**Expected**
- Golden table plus IFS change about −0.03 pp

**Trap**
- Uses Foundry segment total (or IFS segment total for FY2022) despite the explicit prohibition

**What it tests**
- Whether the model drops constraints under pressure (H4) even when they are explicit

---

## 7. How Gemini performed across your runs (summary)
- Early run: partially correct (FY2024 IFS external used correctly) but wrong on FY2022 (used $895M segment revenue) and mixed definitions across years
- Later run: correct use of the external-only series for both years (474, 385) and correct mix math, with only a small rounding variance on the pp change
- Remaining gap: the narrative conclusion (“diversified revenue base”) is often incomplete or not tightly supported by the computed mix

---

## 8. Rubric guardrails (to prevent false passes)

### Must-have checks
- **Line item pinning**
- Must explicitly cite or name the “external customers” series for IFS
- Must not use Intel Foundry / IFS segment totals as the IFS numerator

- **Apples-to-apples definition**
- Same IFS definition in FY2022 and FY2024 (no mixing segment totals in one year and external series in the other)

- **Sanity threshold**
- If IFS “external mix” is above ~1% here, require re-check of numerator source

### Rounding policy
- Compute shares from raw dollars first, then compute pp change, then round at the end

---

# Workflow guide: How to create 10+ questions that extrapolate this error mode

## A. Define the “question factory” target
You want companies where:
- A segment metric labeled “revenue” or “net sales” includes intersegment/internal transfers, or
- The segment note includes an “eliminations” line, and
- The filing also discloses an “external customers” revenue series (often in a different table or footnote)

Your goal is to create prompts where:
- The external-only constraint appears once (easy to drop)
- The trap numerator is available in the canonical segment table (easy to select)

---

## B. Search workflow to find similar companies (high recall)
Use these exact search strings:
- `"net sales to external customers" intersegment 10-k`
- `"revenue from external customers" "internal customers" 10-q`
- `"intersegment net sales" "eliminated in consolidation" segment`
- `"elimination of inter-segment sales" "net sales" segment`

Selection criteria:
- Strong traps have large internal/intersegment activity or prominent eliminations
- Best traps place the external-only series outside the main segment revenue table

---

## C. Deep dive workflow per company (repeatable checklist)
For each candidate filing:
1. Open the segment footnote
2. Locate keywords: “external customers”, “intersegment”, “eliminations”, “to other segments”
3. Identify two competing numerators
- Correct numerator: external customers only
- Trap numerator: segment revenue or segment net sales including intersegment
4. Quantify trap strength
- Compute mix using both numerators
- Keep only cases where the delta is meaningfully large (rule of thumb: > 2 pp or order-of-magnitude)
5. Write 1 to 3 questions per company using the templates below
6. Create a rubric that hard-pins the correct line label
7. Run 3 to 5 model trials and log whether it:
- quotes “external customers”
- mentions eliminations
- uses consistent definitions across years

---

## D. Templates that reliably trigger the failure

### Template 1: Mix change (direct analog to Intel)
“Using FY[Year1] and FY[Year2] filings, compute [Segment X] **external customer revenue** as a % of total revenue for each year and the pp change.”

Trap: segment revenue includes intersegment.

### Template 2: Reconciliation and eliminations
“Reconcile segment net sales to consolidated net sales by applying eliminations, then compute external-only mix.”

Trap: model ignores eliminations.

### Template 3: Internal intensity ratio
“For FY[Year], compute internal/intersegment sales as % of segment net sales, and explain why internal sales cannot be used for external revenue mix.”

Trap: model reports only segment total.

---

## E. Starter candidate list (use these to generate 10+ questions)
Each of these is a reasonable target for the same mode, based on segment disclosures that reference external customers and/or intersegment activity.

- Intel (INTC)
- Commercial Metals (CMC)
- Mettler-Toledo (MTD)
- Snap-on (SNA)
- SpartanNash (SPTN)
- TransDigm (TDG)
- Federal Signal (FSS)
- Clearfield (CLFD)
- Advanced Drainage Systems (WMS)
- HEICO (HEI)

Recommendation:
- Create 1 to 2 questions per company and you will exceed 10 quickly

---

## F. 12 ready-to-use prompt ideas (10+)
1. INTC: external foundry revenue share of total revenue in FY2022 vs FY2024, pp change
2. INTC: compare Foundry segment total share vs external-only share, explain why one is invalid for external mix
3. INTC: compute external foundry revenue as % of Foundry segment total for FY2024 (external intensity)

4. CMC: compute segment mix using “net sales to external customers” and show effect of “intersegment eliminations”
5. CMC: compute intersegment net sales as % of segment net sales for each segment

6. MTD: compute “net sales to other segments” as % of segment net sales, and explain implication for mix calculations
7. MTD: compare two periods and compute change in internal sales ratio

8. SNA: compute segment mix excluding intersegment net sales where segment net sales include both external and intersegment

9. SPTN: compute net sales excluding inter-segment sales and quantify eliminations as % of net sales including inter-segment sales

10. CLFD: compute internal-customer revenue as % of consolidated revenue and explain why it must be eliminated

11. WMS: compute segment mix using “net sales to external customers” and verify sum ties to consolidated net sales

12. HEI or TDG: identify whether intersegment sales exist and how they are treated, then compute segment mix using external-only figures

---

## G. Logging and debugging signals (for harnessing)
Track these signals per run:
- Did the model quote “external customers” (or equivalent)
- Did it reference “intersegment” or “eliminations”
- Did it pull the IFS (or target segment) numerator from the segment table row vs the external-only disclosure
- Did it keep the numerator definition consistent across years
- Did it produce a sanity-checked mix (external mix should be small if external-only is small)

---

## Appendix: Links (copy-paste)

Intel FY2024 10-K PDF (SEC)
https://www.intc.com/filings-reports/all-sec-filings/content/0000050863-25-000009/0000050863-25-000009.pdf

Intel filings page
https://www.intc.com/filings-reports/all-sec-filings/xbrl_doc_only/3812

Commercial Metals (CMC) 10-Q HTML (example)
https://www.sec.gov/Archives/edgar/data/22444/000002244425000013/cmc-20241130.htm

Commercial Metals (CMC) 10-Q HTML (example 2)
https://www.sec.gov/Archives/edgar/data/22444/000002244425000098/cmc-20250531.htm

Mettler-Toledo (MTD) 10-Q HTML (example)
https://www.sec.gov/Archives/edgar/data/1037646/000103764625000021/mtd-20250331.htm

Mettler-Toledo (MTD) 10-Q HTML (example 2)
https://www.sec.gov/Archives/edgar/data/1037646/000103764624000027/mtd-20240630.htm

Snap-on FY2024 10-K PDF
https://www.snapon.com/Snap-on-Files/Investors/2024-Q4/SNA-FY24-10K-Document_Final-Filed.pdf

SpartanNash SEC filings page
https://corporate.spartannash.com/sec-filings?action=view&filer=Ticker%3ASPTN&item=960871&pagetemplate=basic

TransDigm (TDG) 10-K HTML (example)
https://www.sec.gov/Archives/edgar/data/1260221/000126022125000014/tdg-20241228.htm

Federal Signal 10-Q PDF (example)
https://www.federalsignal.com/hubfs/Federal%20Signal/Quarterly%20and%20Annual%20Reports/FSS-2025.09.30-10Q.pdf

Clearfield 10-Q PDF (example)
https://ir.seeclearfield.com/sec-filings/all-sec-filings/content/0001171843-25-005167/0001171843-25-005167.pdf

Advanced Drainage Systems (WMS) FY2022 10-K HTML
https://www.sec.gov/Archives/edgar/data/1604028/000160402822000025/wms-20220331.htm

HEICO annual report on Form 10-K PDF (example)
https://www.sec.gov/Archives/edgar/data/46619/000114036125002565/ny20040474x3_ars.pdf