Fetch Firestore Object Utility

Overview

The fetch-firestore-object.sh script provides a simple command-line interface to fetch and inspect Firestore documents for debugging and development purposes.

Script Location: cli/sdlc/utils/fetch-firestore-object.sh

Purpose

This utility simplifies debugging by allowing developers to quickly inspect Firestore documents without writing custom code or using the Firebase Console. It's particularly useful for:

• ✅ Debugging task progress - Inspect background task status and progress
• ✅ Verifying user data - Check user profiles and permissions
• ✅ Inspecting project metadata - View project settings and configuration
• ✅ Troubleshooting RBAC - Examine access control lists and permissions
• ✅ Development testing - Verify data structure and field values

Usage

Basic Syntax

./cli/sdlc/utils/fetch-firestore-object.sh <collection> <documentId> [project]

Parameters:

• collection - Firestore collection name (e.g., tasks, users, projects)
• documentId - Document ID to fetch
• project - (Optional) GCP project ID (default: construction-code-expert-test)

Common Examples

Fetch Task Document

# Fetch a task from test environment
./cli/sdlc/utils/fetch-firestore-object.sh tasks a7326bb7-9d32-442c-85bc-0807541f893d

# Fetch from specific environment
./cli/sdlc/utils/fetch-firestore-object.sh tasks a7326bb7-9d32-442c-85bc-0807541f893d construction-code-expert-dev

Output:

🔍 Fetching Firestore document
📁 Project: construction-code-expert-test
📂 Collection: tasks
📄 Document ID: a7326bb7-9d32-442c-85bc-0807541f893d

🌐 Querying: https://firestore.googleapis.com/v1/projects/...

✅ Document found!

📊 Raw Firestore Document:
{
  "name": "projects/construction-code-expert-test/databases/(default)/documents/tasks/a7326bb7-9d32-442c-85bc-0807541f893d",
  "fields": {
    "taskId": {
      "stringValue": "a7326bb7-9d32-442c-85bc-0807541f893d"
    },
    "taskType": {
      "stringValue": "PLAN_INGESTION"
    },
    "status": {
      "stringValue": "COMPLETED"
    },
    "progressPercent": {
      "integerValue": "100"
    }
  },
  "createTime": "2024-10-05T10:30:00.123456Z",
  "updateTime": "2024-10-05T10:35:22.789012Z"
}

📋 Document Fields (simplified):
{
  "taskId": {
    "stringValue": "a7326bb7-9d32-442c-85bc-0807541f893d"
  },
  "taskType": {
    "stringValue": "PLAN_INGESTION"
  },
  ...
}

✅ Fetch complete!

Fetch User Document

# Fetch user profile
./cli/sdlc/utils/fetch-firestore-object.sh users john.doe@example.com

# Fetch from dev environment
./cli/sdlc/utils/fetch-firestore-object.sh users john.doe@example.com construction-code-expert-dev

Use Case: Verify user permissions, custom claims, or profile data.

Fetch Project Document

# Fetch project metadata
./cli/sdlc/utils/fetch-firestore-object.sh projects A.0155.01-2025-06-12

# Fetch ACL (Access Control List)
./cli/sdlc/utils/fetch-firestore-object.sh acl A.0155.01-2025-06-12

Use Case: Debug project settings, RBAC permissions, or metadata.

Fetch Custom Collections

# Fetch from any collection
./cli/sdlc/utils/fetch-firestore-object.sh notifications user-123-notification-456
./cli/sdlc/utils/fetch-firestore-object.sh settings global-config
./cli/sdlc/utils/fetch-firestore-object.sh analytics event-2024-10-05

Common Use Cases

1. Debug Background Task Progress

When a background task (plan ingestion, code applicability, compliance report) is stuck or showing incorrect progress:

# Get task ID from UI or logs
TASK_ID="a7326bb7-9d32-442c-85bc-0807541f893d"

# Fetch task document
./cli/sdlc/utils/fetch-firestore-object.sh tasks $TASK_ID

# Check fields:
# - status: PENDING, IN_PROGRESS, COMPLETED, FAILED
# - progressPercent: 0-100
# - currentStep: Current step name
# - errorMessage: Error details if failed

What to Look For:

• ✅ status field - Is it stuck in IN_PROGRESS?
• ✅ progressPercent - Is it updating?
• ✅ currentStep - Which step is it on?
• ✅ updateTime - When was it last updated?
• ✅ errorMessage - Any error details?

2. Verify User Permissions

When debugging RBAC issues or user access problems:

# Fetch user document
./cli/sdlc/utils/fetch-firestore-object.sh users user@example.com

# Check fields:
# - customClaims: Firebase custom claims
# - roles: User roles (if stored in Firestore)
# - projects: Projects user has access to

What to Look For:

• ✅ Custom claims match expected roles
• ✅ Project permissions are correct
• ✅ User is not disabled or restricted

3. Inspect Project Access Control

When debugging project sharing or permission issues:

# Fetch project ACL
PROJECT_ID="A.0155.01-2025-06-12"
./cli/sdlc/utils/fetch-firestore-object.sh acl $PROJECT_ID

# Check fields:
# - owner: Project owner email
# - editors: List of users with EDITOR role
# - viewers: List of users with READER role
# - prompters: List of users with PROMPTER role

What to Look For:

• ✅ User is in the correct role list
• ✅ Owner is set correctly
• ✅ No duplicate entries

4. Verify Data Migration

After data migration or backfill operations:

# Fetch migrated document
./cli/sdlc/utils/fetch-firestore-object.sh projects LEGACY-PROJECT-ID

# Verify:
# - All expected fields are present
# - Data types are correct
# - No missing or null values

5. Troubleshoot Integration Tests

When integration tests fail due to data issues:

# Fetch test data
./cli/sdlc/utils/fetch-firestore-object.sh tasks TEST-TASK-ID construction-code-expert-test

# Verify test data setup
./cli/sdlc/utils/fetch-firestore-object.sh projects TEST-PROJECT-ID construction-code-expert-test

Output Format

The script provides two views of the document:

1. Raw Firestore Document

Complete Firestore REST API response including:

• name - Full document path
• fields - All document fields with Firestore type wrappers
• createTime - When document was created
• updateTime - When document was last modified

Example:

{
  "name": "projects/.../documents/tasks/abc123",
  "fields": {
    "taskId": {
      "stringValue": "abc123"
    },
    "progressPercent": {
      "integerValue": "75"
    },
    "isComplete": {
      "booleanValue": true
    }
  },
  "createTime": "2024-10-05T10:30:00.123456Z",
  "updateTime": "2024-10-05T10:35:22.789012Z"
}

2. Simplified Fields View

Just the fields object for easier reading:

{
  "taskId": {
    "stringValue": "abc123"
  },
  "progressPercent": {
    "integerValue": "75"
  },
  "isComplete": {
    "booleanValue": true
  }
}

Firestore Data Types

Firestore uses type wrappers in the REST API. Here's how to read them:

Firestore Type	JSON Representation	Example
String	{"stringValue": "..."}	{"stringValue": "COMPLETED"}
Integer	{"integerValue": "..."}	{"integerValue": "100"}
Double	{"doubleValue": ...}	{"doubleValue": 3.14}
Boolean	{"booleanValue": ...}	{"booleanValue": true}
Timestamp	{"timestampValue": "..."}	{"timestampValue": "2024-10-05T10:30:00Z"}
Array	{"arrayValue": {"values": [...]}}	{"arrayValue": {"values": [{"stringValue": "a"}, {"stringValue": "b"}]}}
Map	{"mapValue": {"fields": {...}}}	{"mapValue": {"fields": {"key": {"stringValue": "value"}}}}
Null	{"nullValue": null}	{"nullValue": null}

Error Handling

Document Not Found

$ ./cli/sdlc/utils/fetch-firestore-object.sh tasks DOES-NOT-EXIST

🔍 Fetching Firestore document
📁 Project: construction-code-expert-test
📂 Collection: tasks
📄 Document ID: DOES-NOT-EXIST

🌐 Querying: https://firestore.googleapis.com/v1/...

❌ Document not found or access denied
{
  "error": {
    "code": 404,
    "message": "Document not found",
    "status": "NOT_FOUND"
  }
}

Solution: Verify the document ID and collection name are correct.

Authentication Error

$ ./cli/sdlc/utils/fetch-firestore-object.sh tasks abc123

Error: (gcloud.auth.print-access-token) You do not currently have an active account selected.

Solution: Authenticate with gcloud:

gcloud auth login
gcloud config set project construction-code-expert-test

Permission Denied

❌ Document not found or access denied
{
  "error": {
    "code": 403,
    "message": "Missing or insufficient permissions",
    "status": "PERMISSION_DENIED"
  }
}

Solution: Ensure you have Firestore read permissions in the target project:

# Check current project
gcloud config get-value project

# Switch to correct project
gcloud config set project construction-code-expert-test

# Verify IAM permissions
gcloud projects get-iam-policy construction-code-expert-test --flatten="bindings[].members" --filter="bindings.members:user:YOUR_EMAIL"

Integration with Other Tools

With query-task-timing.sh

Combine with task timing analysis for comprehensive debugging:

# 1. Fetch task document
./cli/sdlc/utils/fetch-firestore-object.sh tasks $TASK_ID

# 2. Analyze task timing
./cli/sdlc/utils/query-task-timing.sh $TASK_ID

# 3. Compare progress percent with timing data

With jq for Advanced Filtering

Use jq to extract specific fields:

# Get just the status field
./cli/sdlc/utils/fetch-firestore-object.sh tasks $TASK_ID | \
  jq -r '.fields.status.stringValue'

# Get progress percent
./cli/sdlc/utils/fetch-firestore-object.sh tasks $TASK_ID | \
  jq -r '.fields.progressPercent.integerValue'

# Get all string fields
./cli/sdlc/utils/fetch-firestore-object.sh tasks $TASK_ID | \
  jq '.fields | to_entries | map(select(.value.stringValue)) | from_entries'

In Shell Scripts

Use in automated debugging scripts:

#!/bin/bash
# debug-task.sh - Automated task debugging

TASK_ID="$1"

echo "Fetching task document..."
TASK_DOC=$(./cli/sdlc/utils/fetch-firestore-object.sh tasks $TASK_ID)

STATUS=$(echo "$TASK_DOC" | jq -r '.fields.status.stringValue')
PROGRESS=$(echo "$TASK_DOC" | jq -r '.fields.progressPercent.integerValue')

echo "Task Status: $STATUS"
echo "Progress: $PROGRESS%"

if [[ "$STATUS" == "FAILED" ]]; then
  ERROR=$(echo "$TASK_DOC" | jq -r '.fields.errorMessage.stringValue')
  echo "Error: $ERROR"
fi

Best Practices

1. Use Appropriate Environment

Always specify the correct project for the environment you're debugging:

# Development
./cli/sdlc/utils/fetch-firestore-object.sh tasks $TASK_ID construction-code-expert-dev

# Test
./cli/sdlc/utils/fetch-firestore-object.sh tasks $TASK_ID construction-code-expert-test

# Production (use with caution)
./cli/sdlc/utils/fetch-firestore-object.sh tasks $TASK_ID construction-code-expert-prod

2. Combine with Logs

Cross-reference Firestore data with application logs:

# 1. Fetch task document
./cli/sdlc/utils/fetch-firestore-object.sh tasks $TASK_ID

# 2. Check Cloud Run logs for same task
gcloud logging read "resource.type=cloud_run_revision AND jsonPayload.taskId=$TASK_ID" \
  --limit 50 \
  --format json

3. Document Your Findings

When debugging, document what you find:

# Save output for later analysis
./cli/sdlc/utils/fetch-firestore-object.sh tasks $TASK_ID > task-debug-$(date +%Y%m%d-%H%M%S).json

# Add notes
echo "Task stuck at 75% - investigating step timing" >> task-debug-notes.txt

4. Respect Data Privacy

Be mindful of sensitive data when sharing debug output:

# Sanitize output before sharing
./cli/sdlc/utils/fetch-firestore-object.sh users $USER_ID | \
  jq 'del(.fields.email, .fields.phoneNumber)'

Prerequisites

• gcloud CLI - Must be installed and authenticated
• jq - JSON processor (for parsing output)
• curl - HTTP client (usually pre-installed)
• Firestore permissions - Read access to target project

Installation

# Install gcloud CLI
# https://cloud.google.com/sdk/docs/install

# Install jq
# macOS
brew install jq

# Ubuntu/Debian
sudo apt-get install jq

# Authenticate
gcloud auth login
gcloud config set project construction-code-expert-test

Troubleshooting

jq Command Not Found

# Install jq
brew install jq  # macOS
sudo apt-get install jq  # Ubuntu/Debian

Wrong Project Selected

# Check current project
gcloud config get-value project

# Switch project
gcloud config set project construction-code-expert-test

Rate Limiting

If you're fetching many documents rapidly:

# Add delay between requests
for id in $TASK_IDS; do
  ./cli/sdlc/utils/fetch-firestore-object.sh tasks $id
  sleep 1  # Wait 1 second between requests
done

See Also

• Query Task Timing Utility - Analyze task performance
• Developer Playbook
• Background Tasks Architecture
• RBAC Documentation