How to Build an AI Browser Agent That Completes Tasks Like a Human

Traditional web scraping and browser automation rely on static CSS selectors or XPath paths. When a website redesigns their page layouts, changes class names, or updates button positions, these scripts fail instantly.

AI Browser Agents solve this fragility by combining browser automation tools (like Playwright) with LLM visual and text reasoning. The agent is directed by high-level goals (e.g., "Find the cheapest flight") rather than rigid mouse click paths.

1. Choosing Your Browser Agent Stack

In 2026, the two primary frameworks for building AI browser agents are:

• Stagehand (TypeScript): A CDP (Chrome DevTools Protocol) library that offers surgical primitives like .act(), .extract(), and .observe(). It is highly predictable and designed to handle resilient selector parsing.
• Browser-Use (Python): An autonomous agent loop where the model receives page screenshots and DOM trees, plans its next action, and interacts with the page in a continuous loop.

Both run on top of Playwright as their underlying browser execution engine.

2. Coding Guide: Building with Stagehand (TypeScript)

Let's build a script that logs into a portal and extracts invoice details.

Step 2.1: Initialize Stagehand

Install the package:

npm install @browserbase/stagehand

Step 2.2: Implement the Automation Script

import { Stagehand } from "@browserbase/stagehand";
import os from "os";
 
async function runBrowserAgent() {
  const stagehand = new Stagehand({
    env: "LOCAL",
    apiKey: process.env.STAGEHAND_API_KEY,
    // Connect Stagehand's model router to Weavecode's endpoint
    modelClientOptions: {
      baseURL: "https://api.weavecode.ai/v1",
      apiKey: process.env.WEAVECODE_KEY,
    }
  });
 
  await stagehand.init();
  const page = stagehand.page;
 
  // 1. Navigate to the portal
  await page.goto("https://portal.example.com/login");
 
  // 2. Perform natural language browser actions
  await page.act({ action: "Enter username 'admin' and password 'Secret123'" });
  await page.act({ action: "Click the sign-in button" });
 
  // 3. Navigate to billing
  await page.goto("https://portal.example.com/dashboard/billing");
 
  // 4. Extract structured details dynamically
  const invoices = await page.extract({
    instruction: "Extract the list of all invoices showing date, invoice number, and status.",
    schema: {
      type: "array",
      items: {
        type: "object",
        properties: {
          invoice_id: { type: "string" },
          date: { type: "string" },
          amount: { type: "string" },
          status: { type: "string" }
        },
        required: ["invoice_id", "amount", "status"]
      }
    }
  });
 
  console.log("Extracted Invoices:", invoices);
  await stagehand.close();
}
 
runBrowserAgent();

3. Best Practices for Production

1. Proxy Rotation: Always use residential proxies to bypass bot-detection algorithms on protected sites.
2. Handle CAPTCHAs: Integrate solver services (like Capsolver) directly inside the Playwright request pipeline.
3. Session Re-use: Store authenticated cookies locally to avoid triggering multi-factor login checks on every script execution.