Juno — AI Portfolio Assistant

Why I Built Juno#

Most portfolio chatbots are simple LLM wrappers that only repeat whatever text is hardcoded in a static prompt. If a visitor asks about a specific repository, library version, or code snippet, the bot either guesses or hallucinates.

I wanted to build an assistant right into this portfolio that could actually inspect my work live. I designed Juno as an in-memory Agentic RAG (Retrieval-Augmented Generation) system. When someone asks about my projects or coding background, Juno retrieves relevant snippets from an in-memory vector matrix across all my case studies, searches my public GitHub repositories, inspects READMEs, reads actual source files, and checks my live commits before answering.

You can try it anytime by clicking the chat icon in the bottom-right corner of this site.

How It Works#

Juno runs on a custom streaming agent loop built inside a Next.js 14 App Router API route (app/api/chat/route.ts).

Visitor Message (Browser)
          │
          ▼
┌────────────────────────────────────────┐
│     Next.js 14 App Router Endpoint     │ (app/api/chat/route.ts)
└──────────────────┬─────────────────────┘
                   │
                   ▼
┌────────────────────────────────────────┐
│   Upstash Redis Rate Limiting Guard    │ (20 req/min & 200 req/day per IP)
└──────────────────┬─────────────────────┘
                   │
                   ▼
┌────────────────────────────────────────┐
│   Multi-Model Agent Loop & Cascade     │ (app/api/chat/route.ts & lib/chat/agent.ts)
│  ├── Primary: Gemini 3.1 Flash Lite    │
│  ├── 12s Fast Failover & 60s Cooldown  │
│  ├── Fallbacks: Gemini 3 & 3.5 Flash   │
│  ├── Multi-Turn Tool Execution         │
│  └── Thought Signature Preservation    │
└──────────────────┬─────────────────────┘
                   │
         Tool Call Needed?
         ├── search_knowledge (Vector search over case studies & repos)
         ├── get_recent_activity (Live commits & push events)
         ├── read_case_study (Deep dive into full case study)
         ├── search_github / read_github_file (Inspect code files)
         ├── web_search (Tavily search)
         └── email_follow_up (Direct message)
                   │
                   ▼
┌────────────────────────────────────────┐
│     SSE Streaming to Chat Interface    │ (Live text tokens & tool status chips)
└────────────────────────────────────────┘

The Request Flow

1. 01
   Rate Limiting: When a message arrives, Upstash Redis checks the visitor's IP against two sliding windows: 20 requests per minute and 200 requests per day. This prevents bots from draining API credits.
2. 02
   Context & Knowledge Search: The server injects my verified background (data/profile.ts), recent repo metadata, and available tool schemas into the system prompt. Juno also has access to an in-memory vector matrix of all 13 case studies and 61 public repositories.
3. 03
   Resilient Agent Execution: The route executes a priority model cascade starting with gemini-3.1-flash-lite. If Google's API returns a 503 capacity overload or encounters a connection timeout, the engine instantly engages an in-memory circuit breaker and switches to healthy fallback models (gemini-3-flash-preview, gemini-3.5-flash) without dropping the visitor's connection. If specific details are needed, Juno calls tools (like search_knowledge, get_recent_activity, or read_github_file) for up to 4 iterative rounds.
4. 04
   SSE Streaming: Chunks stream to the browser as Server-Sent Events (data: {"text": "..."}). When a tool is executing, the server sends a status event (data: {"status": "Searching engineering knowledge base..."}) so the interface never feels frozen.

Technical Hurdles I Solved#

In-Memory Agentic RAG Without an External Database

I wanted Juno to answer specific technical questions about my projects — like how I handled WebSockets in my chat app, OCR in legal documents, or composite B-tree indexing in database queries — without having to read through whole case studies one by one.

At first, setting up an external vector database like Pinecone or Supabase seemed like the obvious choice. But free cloud databases come with frustrating trade-offs: they pause or sleep after a week without traffic, and every vector query adds another 200ms network round-trip.

To keep things fast and free forever, I built an in-memory RAG pipeline. I precomputed vector embeddings for all 13 case studies and 61 public GitHub repositories at build time using Gemini's embedding model. I saved the unit-length vectors into a lightweight JSON file in the project. When someone asks a question, Juno embeds the query and calculates cosine similarity in Node.js memory with a fast dot product. The search runs across all sections and repos in less than 0.2 milliseconds, with zero database sleep, zero monthly bills, and zero external network hops.

Keeping Live Activity Fresh While Dodging Rate Limits

Visitors often ask what I am working on right now or what my latest commit was. But calling the GitHub API on every visitor question can quickly hit the 60 requests-per-hour limit on unauthenticated serverless IPs.

To fix this, I built a get_recent_activity tool with a 15-minute in-memory cache. It checks my public GitHub events, picks out the commit messages, branches, and repository names, and formats them into a clean timeline. Visitors get live, accurate answers about what I coded today, while the GitHub API is queried at most once every quarter-hour.

Automated Model Cascade with Circuit-Breaker Self-Healing

During peak usage hours, Google's Gemini v1beta endpoint occasionally returns HTTP 503 UNAVAILABLE with the notice: "This model is currently experiencing high demand. Spikes in demand are usually temporary."

In a production chatbot, failing the user's stream during a temporary 30-second upstream spike creates a jarring experience. To eliminate downtime, I engineered an automated multi-model cascade with an in-memory circuit breaker:

• Priority Cascade & Fast Timeouts: Configured with primary (gemini-3.1-flash-lite) and resilient fallback (gemini-3-flash-preview, gemini-3.5-flash, gemini-3.5-flash-lite) models. The OpenAI client enforces a strict 12-second per-turn timeout with zero internal retries, guaranteeing that any upstream Google stall triggers an immediate model switch well before serverless function execution limits.
• Circuit-Breaker Cooldown: When a model hits a 503 overload or 429 quota spike, it is placed on an in-memory cooldown (configurable via LLM_COOLDOWN_SECONDS, defaulting to 60s). Any subsequent questions in that same window immediately skip straight to the healthy fallback model, saving 200ms of failed round-trip latency.
• Zero-Flicker UI Recovery: Before engaging the fallback model, the server emits an SSE {"reset": true} event, instructing the frontend widget to clear any half-transmitted draft tokens so the fallback stream starts cleanly.
• Automatic Self-Healing: Because the cooldown is timestamp-based, the moment 60 seconds elapse, the very next question automatically probes the primary model again. As soon as Google's spike clears, traffic automatically switches back to the primary model without requiring a server reboot or configuration changes.

Preserving Gemini 3 Thought Signatures

When I upgraded the bot to Gemini 3, tool calls started failing with cryptic 400 Bad Request errors. Google Gemini 3 models produce internal reasoning that includes an extra_content.google.thought_signature object on tool calls. If your code strips this object when returning tool results back to the model, the API rejects the request. I updated lib/chat/agent.ts to preserve and pass this signature through every round.

Keeping Streaming Responsive During Tool Calls

Running tool calls mid-stream can take 1 to 2 seconds while the server talks to the GitHub API. If the connection stays silent during that time, visitors think the bot crashed. I added dedicated SSE event types (status and reset) that render small animated status badges in the chat window, letting users know exactly what the assistant is doing.

File Security & Privacy

Because Juno has tools that read repo files, I added path sanitization to block access to sensitive files like .env, credentials, private keys, and binary assets. It can only read text files, markdown, and notebooks from public repositories.

Takeaways#

• Status feedback makes latency tolerable: Giving the user visible progress updates during multi-step tool calls makes the wait feel intentional and trustworthy rather than broken.
• Strict rate limits are essential for public AI endpoints: A simple sliding-window check in Redis prevents unintended loops and keeps API bills completely predictable.
• In-memory vectors beat cloud databases for small portfolios: If your documents change only when you deploy, precomputing embeddings at build time gives you instant sub-millisecond search with zero database maintenance.