At most one email a week. Unfollow anytime with one click.
Transcript
The full narration of this video. The current sentence is highlighted while the video plays; click any sentence to jump there.
Show transcript
Introduction
Welcome back. In the previous episode, AI searched and read. Today AI starts using the browser the way we do: opening pages, clicking buttons, and filling in fields. In episode seventeen, we meet browser agents such as Claude in Chrome and ChatGPT Agent, watch how they work step by step through the log, and learn to give them clear and safe tasks.
Step 1: What a browser agent is
Imagine having an assistant sitting at your laptop. You just say: please check the price at this store and add it to the cart. Then it opens the browser and does it. That's a browser agent: AI that uses a browser like a person. It looks at the page, decides a step, then clicks, types, and scrolls on its own until the task is done. This is different from the automations we'll build later in Session 6. Automations talk to apps through special channels behind the scenes. A browser agent works in front of the screen, through the same view you see. That's why it can be used on almost any site, including ones with no integrations. It works as a cycle. First, the agent looks at the screen. It reads the page layout, much like we look at a screenshot. Second, it thinks: with a screen like this, what's the most sensible step toward the goal? Third, it acts: clicking a button, typing in a field, or scrolling down. One small action at a time, just like our hand on the mouse and keyboard. Fourth, it checks the result. Did the page change as expected? Then the cycle repeats. For one task, this cycle can happen dozens of times. That's why a browser agent is slower than a person who knows the way by heart. Let's line it up with the previous episode. Web search searches and summarizes. Deep research researches and writes reports. A browser agent opens pages, clicks, and fills things in. The most important difference is in the second row. Web search and deep research only read. A browser agent can press buttons. On time, a browser agent needs a few minutes, depending on how long the task is. Web search finishes in seconds, deep research in minutes, and a browser agent also in minutes, because it moves one step at a time. And because it can press buttons, the risk is higher. It can change things: fill in a form, add to a cart, even send something. That's why the next episode is dedicated to action limits and approval. What do these tasks look like at work? Laras from Sunset Coffee could ask an agent to collect one-liter coffee prices from several online shops. We'll practice this in episode nineteen. Dewi, operations manager at Maju Bersama Co., could ask an agent to fill a shopping cart with office supplies, then stop before payment. Mr. Hendra from Cerdas Tutoring could ask an agent to open a competition organizer's site and summarize the registration schedule. And Nadia could ask an agent to log public information from her clients' competitors' accounts into a table. The pattern is the same: boring click-by-click work that still needs care.
Step 2: Two examples: Claude in Chrome and ChatGPT Agent
Now the two examples people talk about most. Claude in Chrome from Anthropic is an extension in the Chrome browser. ChatGPT Agent is an agent mode inside ChatGPT from OpenAI. Other products, including from Google, are developing similar features. The main difference is the browser used. Claude in Chrome works in your own browser, in the tab you're looking at. ChatGPT Agent works in ChatGPT's virtual browser in the cloud, and you watch its screen from the chat window. As a result, logins work differently too. Claude in Chrome can use sites you're already signed in to in your browser. Convenient, but it means the agent also sees your accounts. In ChatGPT Agent, you can take over its screen to type the login yourself. Both generally ask for permission or confirmation before risky actions, such as sending or paying. But don't rely on that entirely. You should also write the limits into the task yourself. Before trying, there are some practical things to check. First, browser agent availability depends on your subscription plan. Not every plan gets it. Second, some features are limited by region, or still in testing and rolling out gradually. So a friend might already have it while you don't yet. Third, usage limits can change at any time. I'm not giving numbers here because they change quickly. Check the latest pricing page. Fourth, always read Anthropic's and OpenAI's official pages before installing an extension or turning on agent mode. Avoid copycat extensions from unclear sources. Getting started is roughly the same for both. First, turn it on: install the official extension, or choose agent mode from the chat's tools menu. The menu location may differ slightly because AI apps are updated often. Second, set permissions. Where available, decide which sites the agent may open. The narrower, the safer. For practice, start with one or two public sites you know well. Third, give it a clear task: goal, limits, and report format. We'll go over the pattern shortly. Fourth, watch its log, the record of steps the agent writes as it works. If it looks like it's going the wrong way, just stop it. You're always in control.
Step 3: Watching an agent work step by step
Let's follow one complete example. Dewi from Maju Bersama Co. needs to buy paper for the office. She writes the task: at her usual wholesale store, find A4 eighty gram copy paper, pick the cheapest one with enough stock, add ten reams to the cart, and don't check out. The agent doesn't click right away. It writes its plan first: open the store, search for the paper, compare price and stock, add it to the cart, then stop before payment. A plan like this is your first chance to correct things. If something's wrong, like the wrong paper size, fix it now before the agent starts working. The store name and prices in this example are fictional. Now let's look at the screen. On the left, the web page the agent opened. On the right, its step log. The first line of the log is the plan that was just approved. Step one, the agent opens the wholesale store's site. It looks at the page layout and recognizes a search box and a search button, just as we would. Then it clicks the search box. Notice the cursor marker on the screen: that's where the agent clicked. The agent types the keywords: A4 eighty gram copy paper. The field is now filled, marked in blue. Then the agent clicks the search button. Five log lines, five small steps, and you can review every one of them. The search results page opens, with three products. This is where a person usually starts comparing carefully, and the agent does the same. The agent reads the name, stock, and price of each product, just like our eyes scanning a shopping list. Look at product three. It's the cheapest, but it turns out to be seventy gram paper, not eighty. The agent skips it because it doesn't match the task. Product two is cheaper than product one, but it only has eight reams in stock, and Dewi needs ten. It's skipped too. In the end the agent picks product one: eighty gram, enough stock, fifty-two thousand rupiah per ream. This is where an agent beats a plain search: it weighs several conditions at once. Then it enters a quantity of ten and clicks add to cart. Notice the log also records reasons, not just actions. From this log, Dewi knows why the other two products weren't chosen. The agent opens the cart page. It holds ten reams of A4 eighty gram paper, a total of five hundred twenty thousand rupiah. Shipping is only calculated at checkout, so it doesn't show here yet. Before reporting, the agent double-checks: the size, paper weight, and quantity all match. This is the check-the-result stage of the cycle. Now notice the red line in the log. The agent stops right before the continue to payment button, as Dewi instructed. A red line marks the point where the agent is waiting for a human. That payment button is clearly visible and could be clicked. What holds the agent back is the limit in the task, plus the app's built-in safeguards. Two layers of protection are better than one. Back in the chat window, the agent writes a short report: what's in the cart, the total, and why the cheapest product was skipped. Dewi reviews the cart, then pays herself. The division of labor is clear: the agent does the boring clicking, and the human keeps the decisions that move money.
Step 4: Reading the agent's step log
From Dewi's example, we learn that the log is a window into the agent's thinking. There are four things to read. First, the plan: does the agent understand its task correctly? Second, the actions: which pages it opened, which fields it filled, which buttons it clicked. From here you can trace its path if a result looks odd. Third, the reasons: why a product was chosen or skipped. Reasons let you judge, not just trust. Fourth, the stop points: when the agent stops and asks for permission. If you're unsure about what's happening, stop the agent anytime, and take over yourself. So when should you stop the agent? Sign one: it opens a site you didn't mention in the task. It may have wandered off through an ad or another link. Two: it repeats the same step over and over, for example clicking the same button while the page doesn't change. That's a sign it's confused. Three: it fills in data you never gave it, like guessing a phone number or an address. The agent is making things up. Four, and most serious: it presses a send or pay button without asking. If you see any of these signs, the action is the same: stop it, clarify the task, then run it again. Usually the problem is instructions that are too loose, not the agent itself. The next step covers how to write a clear task.
Step 5: Writing clear tasks for a browser agent
Now let's take apart the task Dewi wrote. Block one, the goal: an end result you can check. Ten reams of a specific paper are in the cart. Block two, context: who you are and the state of the account. Here Dewi says the store account is already signed in, so the agent doesn't need to figure out how to log in. Block three, the task: the steps and the selection criteria. The cheapest, but with at least ten reams in stock. Clear criteria let the agent weigh options like we saw earlier. Block four, the limits, the most important part for a browser agent. Which sites it may open, which buttons it must not press, and what data it must not save. Block five, the report format: product, price, total, and reason. With a report like this, Dewi can review the result in half a minute. Save this template in your prompt library. Compare a vague task with a clear one. Buy some paper sounds short, but the agent has to guess the size, the quantity, and whether it's allowed to pay. The clear version spells it all out, including don't check out. Find the cheapest one can send the agent wandering everywhere. The clear version limits it to one site, with a minimum stock condition. And just handle the order is a dangerous invitation. The clear version: stop and ask before pressing the pay button. The pattern is simple: every word that makes the agent guess, replace it with a number, a site name, or a clear condition. An agent that doesn't have to guess goes the wrong way far less often. For your first practice, choose safe tasks. Good fits: collecting information from several pages into a table, and comparing products on one site. Also good: filling in a form with data you provide yourself, and checking a status or schedule on a public site. Not a good fit yet, the red crosses: paying or transferring money, and changing account settings or passwords. Keep those two in your own hands.
Step 6: Limitations to accept
To keep your expectations realistic, know its four limitations. First, it's slow. A task you could finish in one minute might take several. The upside: you can do other work while you wait. Second, CAPTCHAs and verifications, the I'm not a robot boxes. The agent usually doesn't solve them on its own, and that's how it should be. Third, site layouts change. A button moves or a promo pop-up appears, and the agent can get confused. A task that ran smoothly yesterday might stall today, so check the results every time. Fourth, the agent can misclick. That's why the log and the approval points we covered aren't a formality; they're the main safeguard. Here's an example. Mr. Hendra asks the agent to summarize competition dates from the organizer's site. The agent opens the site. It turns out the page asks for an I'm not a robot verification. Many sites add safeguards like this precisely to stop automated programs. The agent doesn't try to get around it. It stops, marked by a red line, and asks Mr. Hendra to take over the screen. Mr. Hendra checks the verification box himself, then the agent continues the task. The same take-over pattern is used for logins. You type the password, not the agent. Your assignment before the next episode. First, check on the official page whether your plan and region already have a browser agent. Second, write one browser task for your business with five blocks: goal, context, task, limits, and format. Don't forget the sentence stop before sending or paying in the limits section. Third, if the feature is available, run that task on a public site that doesn't need a login. If not, keep the task; it'll still be useful for the practice in episodes nineteen and twenty. Fourth, read its log and mark the steps where the agent hesitated or went the wrong way. These notes will help you write better limits.
Done!
Let's sum up. A browser agent looks at the page, thinks, acts, and checks the result in a repeating cycle. You control it through a clear task, by reading its log, and by stopping it when it goes the wrong way. In the next episode we tackle the most important question of this session: when AI may act on its own, and when it must ask for your approval. See you in episode eighteen.
Practice exercises
Try these on your own after watching to lock in what you learned.
Check on the official page whether your plan and region have Claude in Chrome, ChatGPT Agent, or a similar agent feature.
Write one browser task for your business using goal, context, task, limits, and format blocks. Include the sentence 'stop before sending or paying'.
If the feature is available, run that task on a public site with no login. Read the log and mark the steps where the agent hesitated or went the wrong way.
Ask the class material
Type a question or keyword; we find the part of this episode that explains it and jump the video there.
My notes
Notes are saved with the current video time and are only visible to you.
Questions & answers
Stuck on a step? Ask here; the instructor and other learners can answer.
No questions yet.
Your instructor
Cerdas Inovasi Digital Team
Software developers, BIM & 3D engineers, and AI practitioners
52 video classes1,276 episodes
We build software, BIM models, games and AI automations for clients every day, and turn that hands-on work into step-by-step classes: every step is shown on screen and comes with a PDF guide you can keep.
