TDD: Chat Response Decorator Agent

Parent Issue: #263 - Agentic Chat
Created: 2025-10-25
Status: Planning

Overview

Create a lightweight post-processing agent that decorates chat responses with interactive hyperlinks, making page references clickable for instant navigation.

Problem Statement

The main chat agent returns helpful responses like:

Here are the files and pages that have information about the Second Floor:

• File 1, Page 6: "SECOND FLOOR PLAN - BLOCK I" (A4.13)
• File 1, Page 7: "SECOND FLOOR BLOCK 2" (A4.14)

But these are plain text - users can't click to navigate. They have to manually:

Read "File 1, Page 6"
Close chat
Navigate file tree → File 1
Scroll to Page 6

Goal: Make page references automatically clickable.

Desired Behavior

Input (from main agent):

File 1, Page 6: "SECOND FLOOR PLAN - BLOCK I" (A4.13)

Output (decorated):

[File 1, Page 6: "SECOND FLOOR PLAN - BLOCK I" (A4.13)](/projects/san-jose-multi-file3/files/1/pages/6/preview)

Rendered in UI:

<a href="/projects/san-jose-multi-file3/files/1/pages/6/preview">
  File 1, Page 6: "SECOND FLOOR PLAN - BLOCK I" (A4.13)
</a>

User clicks → Instantly navigates to File 1, Page 6!

Architecture

High-Level Flow

Main Agent Response (raw text)
    ↓
ChatResponseDecoratorAgent (Gemini 2.0 Flash - fast!)
    ↓
Decorated Response (markdown with [links])
    ↓
Frontend (marked.js renders links)
    ↓
User clicks → Angular Router navigates

Implementation Options

Option A: LLM-Based (Recommended)

Pros: Handles various formats ("Page 6", "page 6", "pg 6", etc.)
Pros: Context-aware (understands intent)
Cons: Small latency (~200ms)
Cost: Minimal (Gemini 2.0 Flash: $0.075/1M input tokens)

Option B: Regex-Based

Pros: Zero latency
Pros: Zero cost
Cons: Brittle (breaks on format variations)
Cons: Requires maintenance for new formats

Recommendation: Start with Option A (LLM), fall back to Option B if latency becomes an issue.

Technical Design

1. Decorator Agent (Java)

File: src/main/java/.../service/ChatResponseDecoratorAgent.java

public class ChatResponseDecoratorAgent {
  
  private static final String MODEL = "gemini-2.0-flash"; // Fast & cheap
  
  /**
   * Decorate chat response with hyperlinks.
   * 
   * @param rawResponse Raw text from main agent
   * @param context Chat context (projectId, fileId, pageNumber)
   * @return Decorated response with markdown links
   */
  public static String decorateResponse(String rawResponse, ChatContext context) {
    
    // Build prompt for decorator agent
    String prompt = buildDecoratorPrompt(rawResponse, context);
    
    // Call Gemini 2.0 Flash (fast single-turn)
    GoogleGenAiClient client = GoogleGenAiClient.builder()
        .setModel(Model.GEMINI_2_0_FLASH)
        .singleTurn()
        .setTemperature(0.0f)
        .build();
    
    String decoratedResponse = client.prompt(prompt);
    
    return decoratedResponse;
  }
  
  private static String buildDecoratorPrompt(String rawResponse, ChatContext context) {
    return String.format("""
        You are a response decorator. Convert page references to markdown links.
        
        Project ID: %s
        
        INPUT TEXT:
        %s
        
        TASK:
        1. Find all page references like "File 1, Page 6", "File 2, Page 3", etc.
        2. Convert to markdown links: [File 1, Page 6](/projects/%s/files/1/pages/6/preview)
        3. Preserve ALL other text exactly as-is
        4. Return ONLY the decorated text (no explanations)
        
        IMPORTANT:
        - Keep all formatting, spacing, and line breaks
        - Only add links, don't change the text
        - URL format: /projects/{projectId}/files/{fileNum}/pages/{pageNum}/preview
        
        OUTPUT (decorated text):
        """,
        context.getProjectId(),
        rawResponse,
        context.getProjectId()
    );
  }
}

2. Integration with ChatAgentService

File: src/main/java/.../service/ChatAgentService.java

public Flux<ChatMessageResponse> processMessage(...) {
  return Flux.create(sink -> {
    // ... existing agent processing ...
    
    // AFTER agent completes, before sending final response:
    String rawResponse = fullResponse.toString();
    
    // Decorate response with links (async, non-blocking)
    String decoratedResponse = ChatResponseDecoratorAgent.decorateResponse(
        rawResponse, 
        context
    );
    
    // Send decorated response instead of raw
    sink.next(ChatMessageResponse.builder()
        .content(decoratedResponse)  // ← Decorated!
        .isFinal(true)
        .build());
  });
}

3. Frontend (No Changes Needed!)

Angular's router already handles relative URLs:

marked.js renders: [File 1, Page 6](/projects/.../pages/6/preview)
Browser renders: <a href="/projects/.../pages/6/preview">File 1, Page 6</a>
User clicks → Angular Router navigates ✅

Example Transformations

Example 1: Simple Page Reference

Input:

File 1, Page 6: "SECOND FLOOR PLAN - BLOCK I"

Output:

[File 1, Page 6](/projects/san-jose-multi-file3/files/1/pages/6/preview): "SECOND FLOOR PLAN - BLOCK I"

Example 2: Multiple References

Input:

The electrical details are on File 2, Page 5 and File 2, Page 8.

Output:

The electrical details are on [File 2, Page 5](/projects/san-jose-multi-file3/files/2/pages/5/preview) and [File 2, Page 8](/projects/san-jose-multi-file3/files/2/pages/8/preview).

Example 3: List Format

Input:

• File 1, Page 6: "SECOND FLOOR PLAN"
• File 1, Page 7: "SECOND FLOOR BLOCK 2"

Output:

• [File 1, Page 6](/projects/san-jose-multi-file3/files/1/pages/6/preview): "SECOND FLOOR PLAN"
• [File 1, Page 7](/projects/san-jose-multi-file3/files/1/pages/7/preview): "SECOND FLOOR BLOCK 2"

Performance

Latency Analysis

Without Decorator:

Main agent: ~2-4 seconds
Total: ~2-4 seconds

With Decorator:

Main agent: ~2-4 seconds
Decorator (Gemini 2.0 Flash): ~200-400ms
Total: ~2.5-4.5 seconds

Impact: +200-400ms (~10% increase)

Cost Analysis

Per decorated response:

Input tokens: ~500 (prompt + response)
Output tokens: ~500 (decorated response)
Cost: $0.000075 per decoration

Monthly cost (1,000 users × 5 queries/day × 30 days):

Total decorations: 150,000
Cost: $11.25/month

Negligible compared to main agent costs (~$30/month).

Implementation Phases

Phase 1: Core Decorator (1 day)

Create ChatResponseDecoratorAgent.java
Implement decorateResponse() method
Write prompt template
Integrate with ChatAgentService
Test with sample responses

Phase 2: Advanced Patterns (1 day)

Handle code section references (e.g., "IBC 2021 Section 1005.3")
Handle detail references (e.g., "Detail 3/A4.1")
Handle cross-references between pages
Add caching for common patterns

Phase 3: Optimization (1 day)

Add response caching (identical responses)
Batch multiple decorations
Performance monitoring
Fallback to regex if LLM unavailable

Alternative Approaches

Approach 1: Regex-Based (No LLM)

public static String decorateResponseRegex(String text, String projectId) {
  // Pattern: "File {num}, Page {num}"
  Pattern pattern = Pattern.compile(
      "File\\s+(\\d+),\\s+Page\\s+(\\d+)", 
      Pattern.CASE_INSENSITIVE
  );
  
  return pattern.matcher(text).replaceAll(match -> {
    String fileNum = match.group(1);
    String pageNum = match.group(2);
    String url = String.format(
        "/projects/%s/files/%s/pages/%s/preview",
        projectId, fileNum, pageNum
    );
    return String.format("[%s](%s)", match.group(0), url);
  });
}

Pros: Zero latency, zero cost
Cons: Misses variations like "page 6", "pg 6", "p. 6"

Approach 2: Hybrid

Try regex first (instant)
If no matches, use LLM (fallback)

Prompt Template

You are a response decorator. Convert page references to markdown links.

PROJECT CONTEXT:
- Project ID: {projectId}
- Current File: {fileId} (optional)
- Current Page: {pageNumber} (optional)

INPUT TEXT:
{rawResponse}

TASK:
1. Find ALL page references in formats like:
   - "File 1, Page 6"
   - "File 2, Page 3: Title"
   - "page 5"
   - "File 1, pages 6-8"

2. Convert to markdown links:
   - Format: [original text](/projects/{projectId}/files/{fileNum}/pages/{pageNum}/preview)
   - Example: [File 1, Page 6](/projects/san-jose-multi-file3/files/1/pages/6/preview)

3. Rules:
   - Preserve ALL text, formatting, line breaks
   - Only add markdown link syntax
   - Don't explain, just return decorated text
   - If uncertain, leave text as-is

OUTPUT (decorated text only):

Testing Strategy

Unit Tests

@Test
public void testDecorateSimpleReference() {
  String input = "File 1, Page 6 has the details.";
  String expected = "[File 1, Page 6](/projects/test-proj/files/1/pages/6/preview) has the details.";
  
  ChatContext context = ChatContext.newBuilder()
      .setProjectId("test-proj")
      .build();
  
  String result = ChatResponseDecoratorAgent.decorateResponse(input, context);
  assertEquals(expected, result);
}

@Test
public void testDecorateMultipleReferences() {
  String input = "See File 1, Page 6 and File 2, Page 3.";
  // Should have 2 links
}

@Test
public void testPreserveFormatting() {
  String input = "• File 1, Page 6\n• File 1, Page 7";
  // Should preserve bullets and line breaks
}

Integration Tests

Send message via chat
Verify response contains markdown links
Verify links have correct project ID
Click link in UI → verify navigation works

Error Handling

Fallback Strategy

public static String decorateResponse(String rawResponse, ChatContext context) {
  try {
    // Try LLM decoration
    return decorateWithLLM(rawResponse, context);
  } catch (Exception e) {
    logger.warning("LLM decoration failed, using regex fallback: " + e.getMessage());
    try {
      // Fallback to regex
      return decorateWithRegex(rawResponse, context.getProjectId());
    } catch (Exception e2) {
      logger.warning("Regex decoration failed, returning original: " + e2.getMessage());
      // Ultimate fallback: return original text
      return rawResponse;
    }
  }
}

Graceful degradation: Always return valid text, even if decoration fails.

Configuration

Environment Variables

# env/{env}/gcp/cloud-run/grpc/vars.yaml
CHAT_DECORATOR_ENABLED: "true"
CHAT_DECORATOR_MODEL: "gemini-2.0-flash"
CHAT_DECORATOR_TIMEOUT_MS: "500"  # Fail fast if slow
CHAT_DECORATOR_FALLBACK: "regex"  # regex | none

Feature Flag

private static final boolean DECORATOR_ENABLED = 
    Boolean.parseBoolean(System.getenv("CHAT_DECORATOR_ENABLED"));

if (DECORATOR_ENABLED) {
  decoratedResponse = ChatResponseDecoratorAgent.decorateResponse(...);
} else {
  decoratedResponse = rawResponse;
}

Link Format Specification

Standard Page Link

Format: /projects/{projectId}/files/{fileNum}/pages/{pageNum}/preview

Example: /projects/san-jose-multi-file3/files/1/pages/6/preview

With Detail Reference (Future)

Format: /projects/{projectId}/files/{fileNum}/pages/{pageNum}/preview#detail-{detailId}

Example: /projects/san-jose-multi-file3/files/1/pages/6/preview#detail-A4.13

Code Section Link (Future)

Format: /icc-codes/{bookId}/sections/{sectionId}

Example: /icc-codes/2217/sections/IBC2021P2_Ch11_Sec1105

Success Metrics

User Experience:

✅ 90%+ of page references become clickable
✅ Click → navigate in <1 second
✅ Correct page loads 95%+ of time

Performance:

✅ P95 decoration latency: <500ms
✅ P99 decoration latency: <1000ms
✅ Fallback success rate: 100%

Reliability:

✅ Never breaks existing functionality
✅ Graceful degradation if LLM unavailable
✅ No increased error rate

Implementation Checklist

Backend (Java)

Testing

Frontend (No Code Changes!)

Verify marked.js renders links correctly
Verify Angular Router handles navigation
Test on mobile (touch targets)
Accessibility audit (screen readers)

Example Decorator Prompt

You are a response decorator. Add markdown links to page references.

PROJECT: san-jose-multi-file3

INPUT:
Here are the files and pages with second floor information:

• File 1, Page 6: "SECOND FLOOR PLAN - BLOCK I" (A4.13)
• File 1, Page 7: "SECOND FLOOR BLOCK 2" (A4.14)

TASK:
Convert page references to markdown links using format:
[File X, Page Y](/projects/san-jose-multi-file3/files/X/pages/Y/preview)

OUTPUT (decorated text only, no explanations):
Here are the files and pages with second floor information:

• [File 1, Page 6](/projects/san-jose-multi-file3/files/1/pages/6/preview): "SECOND FLOOR PLAN - BLOCK I" (A4.13)
• [File 1, Page 7](/projects/san-jose-multi-file3/files/1/pages/7/preview): "SECOND FLOOR BLOCK 2" (A4.14)

Future Enhancements

Phase 2: Smart Links

Detail references: "Detail 3/A4.1" → /projects/.../pages/4/preview#detail-A4-1
Code sections: "IBC 1005.3" → /icc-codes/2217/sections/...
File references: "electrical plan" → /projects/.../files/2/preview

Phase 3: Context-Aware

Relative references: "page 6" (no file number) → infer from context
Range references: "pages 6-8" → link to first page with note
Ambiguous references: Ask for clarification or link to search

Phase 4: Interactive Elements

Quick actions: Hover over link → preview thumbnail
Batch navigation: "Open all 3 pages" button
Pin from link: Right-click link → "Pin this page"

Risks & Mitigations

Risk	Impact	Mitigation
LLM latency `>`1s	Poor UX	500ms timeout, fallback to regex
LLM hallucination	Wrong links	Validate links against actual files
Cost overrun	Budget issue	Monitor usage, add rate limiting
Regex misses formats	Incomplete links	Use LLM as primary, regex as fallback

Estimated Effort

Phase 1 (Core Decorator):

Development: 4 hours
Testing: 2 hours
Documentation: 1 hour
Total: 1 day

Phase 2 (Advanced Patterns):

Development: 4 hours
Testing: 2 hours
Total: 1 day

Phase 3 (Optimization):

Caching: 2 hours
Monitoring: 2 hours
Total: 0.5 day

Grand Total: 2.5 days

Success Criteria

✅ Must Have (Phase 1):

Page references become clickable links
Links navigate to correct page
No increase in errors
<500ms decoration latency

✅ Should Have (Phase 2):

Detail references linkable
Code section references linkable
95%+ accuracy rate

✅ Nice to Have (Phase 3):

Response caching
Batch operations
Preview on hover

Dependencies

✅ Gemini 2.0 Flash API (already have)
✅ marked.js (already installed)
✅ Angular Router (already configured)
✅ ChatContext (already available)

No new dependencies!

Files to Create

Backend (1 new file):

src/main/java/.../service/ChatResponseDecoratorAgent.java (~150 lines)

Backend (1 modified file):

src/main/java/.../service/ChatAgentService.java (add decorator call)

Environment (4 modified files):

env/dev/gcp/cloud-run/grpc/vars.yaml
env/demo/gcp/cloud-run/grpc/vars.yaml
env/test/gcp/cloud-run/grpc/vars.yaml
env/prod/gcp/cloud-run/grpc/vars.yaml

Tests (1 new file):

src/test/java/.../service/ChatResponseDecoratorAgentTest.java (~200 lines)

Deployment

Build

export JAVA_HOME=/usr/lib/jvm/temurin-23-jdk-arm64
mvn clean package -DskipTests

Deploy

cd env/test/gcp/cloud-run/grpc
./deploy.sh

Verify

Ask: "Which pages cover second floor?"
Response should have clickable links
Click link → should navigate to page

Priority: Medium (nice enhancement, not blocking)
Complexity: Low (single-purpose agent, simple integration)
Estimated Effort: 2.5 days
Cost Impact: +$11/month (minimal)

Overview​

Problem Statement​

Desired Behavior​

Input (from main agent):​

Output (decorated):​

Rendered in UI:​

Architecture​

High-Level Flow​

Implementation Options​

Option A: LLM-Based (Recommended)​

Option B: Regex-Based​

Technical Design​

1. Decorator Agent (Java)​

2. Integration with ChatAgentService​

3. Frontend (No Changes Needed!)​

Example Transformations​

Example 1: Simple Page Reference​

Example 2: Multiple References​

Example 3: List Format​

Performance​

Latency Analysis​

Cost Analysis​

Implementation Phases​

Phase 1: Core Decorator (1 day)​

Phase 2: Advanced Patterns (1 day)​

Phase 3: Optimization (1 day)​

Alternative Approaches​

Approach 1: Regex-Based (No LLM)​

Approach 2: Hybrid​

Prompt Template​

Testing Strategy​

Unit Tests​

Integration Tests​

Error Handling​

Fallback Strategy​

Configuration​

Environment Variables​

Feature Flag​

Link Format Specification​

Standard Page Link​

With Detail Reference (Future)​

Code Section Link (Future)​

Success Metrics​

Implementation Checklist​

Backend (Java)​

Testing​

Frontend (No Code Changes!)​

Example Decorator Prompt​

Future Enhancements​

Phase 2: Smart Links​

Phase 3: Context-Aware​

Phase 4: Interactive Elements​

Risks & Mitigations​

Estimated Effort​

Success Criteria​

Dependencies​

Files to Create​

Deployment​

Build​

Deploy​

Verify​

Overview

Problem Statement

Desired Behavior

Input (from main agent):

Output (decorated):

Rendered in UI:

Architecture

High-Level Flow

Implementation Options

Option A: LLM-Based (Recommended)

Option B: Regex-Based

Technical Design

1. Decorator Agent (Java)

2. Integration with ChatAgentService

3. Frontend (No Changes Needed!)

Example Transformations

Example 1: Simple Page Reference

Example 2: Multiple References

Example 3: List Format

Performance

Latency Analysis

Cost Analysis

Implementation Phases

Phase 1: Core Decorator (1 day)

Phase 2: Advanced Patterns (1 day)

Phase 3: Optimization (1 day)

Alternative Approaches

Approach 1: Regex-Based (No LLM)

Approach 2: Hybrid

Prompt Template

Testing Strategy

Unit Tests

Integration Tests

Error Handling

Fallback Strategy

Configuration

Environment Variables

Feature Flag

Link Format Specification

Standard Page Link

With Detail Reference (Future)

Code Section Link (Future)

Success Metrics

Implementation Checklist

Backend (Java)

Testing

Frontend (No Code Changes!)

Example Decorator Prompt

Future Enhancements

Phase 2: Smart Links

Phase 3: Context-Aware

Phase 4: Interactive Elements

Risks & Mitigations

Estimated Effort

Success Criteria

Dependencies

Files to Create

Deployment

Build

Deploy

Verify