How to Build an AI Agent That Actually Handles Boring Tasks for You
Estimated reading time: 13 minutes
- Most AI agents fail due to limitations in interacting with complex websites, unpredictability of LLMs, anti-bot measures, and a lack of common-sense reasoning.
- Building an effective AI agent requires a robust tech stack, particularly the Browser Use library for browser automation and Bright Data’s Agent Browser for human-like web interaction and bypassing anti-bot defenses.
- This tutorial demonstrates crafting an AI agent for job hunting, capable of searching Google, browsing job platforms, filtering listings, extracting details, and exporting them into a clean JSON file.
- Bright Data’s Agent Browser is crucial for making the AI agent “unstoppable” by providing human-like browsing sessions, CAPTCHA solving, IP rotation, and cloud scalability.
- The framework is highly adaptable; by simply changing the task description, the same agent can automate various browser tasks like scheduling flights, tracking prices, or collecting news.
- Why Most AI Agents Don’t Deliver on Their Promise
- Crafting Your Unstoppable AI Agent: A Step-by-Step Tutorial
- Beyond Job Hunting: Automate Any Browser Task
- Final Thoughts
- Frequently Asked Questions
Ah, AI agents… the hottest trend in tech right now. Everyone’s hyped about them being the future of work. After all, they can do it all and will automate most tasks to give us more time, right? Well… sort of.
The reality? Most agents get blocked by websites or get lost while trying to execute tasks. To actually make one that works, you need a best-in-class tech stack. Only the right combination of tools can turn an AI agent into a real task-automation machine.
Follow this tutorial and learn how to craft an AI agent that can truly automate tasks for you!
The dream of having AI automate tasks for us is exactly why AI agents were invented in the first place. It’s why “agentic AI” became a trend, and why the hype is still sky-high. Imagine a world where all the tedious, repetitive stuff gets handled by AI so we can save time. Sounds perfect, right? That way, we could focus on what really matters: stacking V-Bucks in Fortnite or grinding runes in Elden Ring.
Jokes aside, if you’ve ever played around with an AI agent like OpenAI Operator or tried building one yourself, you already know the sad truth: AI agents rarely live up to expectations!
Why Most AI Agents Don’t Deliver on Their Promise
The promise of AI agents is compelling: a tireless digital assistant handling mundane tasks while you focus on creativity, strategy, or just enjoying life. Yet, the current landscape is often filled with frustration. Many developers quickly discover that building an AI agent capable of navigating the real web is far more challenging than it appears on paper. These are some of the main reasons AI agents flop:
- They can’t interact with websites or desktop apps like a real human would. Traditional automation often relies on predictable elements, but AI needs to perceive and adapt, which is difficult without robust browser control.
- LLMs powering them can be unpredictable, giving different results on the same input. The inherent variability of large language models means consistent task execution is a significant hurdle.
- Even when they do use a browser, anti-bot techniques like CAPTCHAs stop them cold. Modern websites are designed to detect and block automated access, turning many AI agent attempts into dead ends.
- Unlike humans, AI agents often lack common sense reasoning and struggle to adapt when faced with situations beyond their programming. A slight change in a website’s layout or an unexpected pop-up can derail an agent that lacks human-like adaptability.
The problem isn’t the idea of AI agents. Instead, it’s the tech stack you use to build them. So let’s stop wasting time and figure out how to build an AI agent that can actually automate browser tasks for you.
Crafting Your Unstoppable AI Agent: A Step-by-Step Tutorial
In this chapter, you’ll be walked through building an AI agent that can handle one of the most boring (yet critical) tasks out there: job hunting! The resulting AI agent will be smart enough to:
- Visit Google
- Discover job platforms
- Browse listings based on your desired positions and preferences
- Extract interesting jobs
- Export them into a clean JSON file
And if you want to take it further, you’ll also find resources on how to feed it your CV so the agent can learn your profile and automatically apply to the best matches—all without you lifting a finger.
Important: This is just an example! As mentioned before the end of this guide, the same agent can be adapted to almost any browser-based workflow by simply changing the task description.
Actionable Step 1: Lay the Foundation with Browser Automation and LLM Integration
To begin, we need to ensure our environment is ready and equip our agent with the fundamental tools for thinking and interacting with web pages. This involves setting up Python, installing a powerful browser automation library, and connecting a Large Language Model (LLM).
Prerequisites:
- An LLM API key (we’ll use Gemini, since it’s basically free to use via API, but OpenAI, Anthropic, Ollama, Groq, and others work as well).
- A Bright Data account with the Browser API enabled (don’t worry about setup yet, as you’ll be guided through it in this tutorial).
- Python ≥ 3.11 installed locally.
To speed things up, we’ll also assume you already have a Python project set up with an uv virtual environment in place.
Install Browser Use:
As mentioned earlier, most AI agents flop because they hit the wall of tech limitations. The models alone just aren’t enough. So what’s one of the best tools to build AI agents that can indeed do stuff inside a browser? Browser Use!
Never heard of it? No worries! Catch up with this video or take a look at its official docs:
First things first, activate your uv venv and install the browser-use package from PyPI:
uv pip install browser-useUnder the hood, this library runs on Playwright, so you’ll also need to grab the Chromium binaries it depends on. To do so, run:
uvx playwright install chromium --with-deps --no-shellBoom! You’re now set up with a browser automation agentic AI powerhouse.
Integrate the LLM:
AI agents won’t do much without AI (shocker, right?), so your agent needs a language model to properly think. Browser Use supports a long list of LLM providers, but we’ll focus on Gemini, the one highlighted on the official browser-use GitHub page. Why Gemini? Because it’s one of the few LLMs with API access and generous rate limits that make it fundamentally free to play with.
Grab your Gemini API key and store it in a .env file in your project folder like this:
GEMINI_API_KEY=<YOUR_GEMINI_API_KEY>Next, create an agent.py file, which will contain the AI agent definition logic. Start by reading the envs from .env using python-dotenv (which comes with browser-use):
from dotenv import load_dotenv # Read the environment variables from the .env file
load_dotenv()Then, define your LLM integration:
from browser_use import ChatGoogle # The LLM powering the AI agent
llm = ChatGoogle(model="gemini-2.5-flash")Amazing! You’ve got your AI engine ready.
Actionable Step 2: Define the Browser-Based Task and Run the Agent
With our foundation set, the next critical step is to clearly define what we want our AI agent to accomplish. The effectiveness of your agent hinges on how precisely you articulate its task, enabling the LLM to understand and execute complex workflows. After defining the task, we’ll run the agent to see its initial capabilities and identify where it might encounter common web automation challenges.
Describe the Browser-Based Task to Automate:
How you describe the task to your agent is everything. The LLM you configured in Browser Use only works as well as your instructions, so spend time crafting a prompt that’s clear, detailed, but not overly complicated. This is the most important step in your implementation. Thus, check out guides on prompt design and follow the Browser-Use best practices to maximize results. You might need a few rounds of trial and error.
Since this is just an example, let’s keep it simple and describe the browser job-hunting task like this:
task = """
Search on Google for software engineer jobs in New York.
1. Choose a job posting page.
2. On the chosen site, filter for jobs published within the last 24 hours.
3. For each job listing, extract the key details, including the job posting URL and the apply URL (if available).
4. Return all results as a JSON list.
"""As you can see, you’re giving your agent a lot of freedom, which is totally fine considering how capable and flexible Browser Use is!
Tip: In a real-world setup, you should read preferences from a configuration file and inject them into your prompt. This makes your agent customizable for different searches. Think varying job titles, locations, required skills, company preferences, remote vs on-site, and more. For a similar approach, read our guide on building a LinkedIn job hunting AI assistant.
Define and Run the Agent:
Use Browser Use to spin up an AI agent controlled by your configured LLM that can tackle the task you defined earlier:
from browser_use import Agent agent = Agent( llm=llm, task=task,
)Fire your agent like this:
history = agent.run_sync()Perfect! Now all that’s left is to grab the output from your AI agent and export it to JSON (or any format you need).
Export the Output to JSON:
Grab the output from your agent (which should be a clean JSON list of jobs) and dump it to a .json file:
import json output_data = history.structured_output
with open("jobs.json", "w", encoding="utf-8") as f: json.dump(output_data, f, ensure_ascii=False, indent=4)Here we go! Mission complete. Boring task handler agent at your service!
Actionable Step 3: Empower Your Agent with Human-Like Browsing Capabilities
While the previous steps give us a functional AI agent, it’s likely to hit a wall when faced with real-world website defenses. This is where most AI agents falter. To make our agent truly unstoppable, we need to integrate a sophisticated browser solution that can mimic human behavior, bypass anti-bot measures, and ensure uninterrupted task execution. This final step is the game-changer.
Address the Agent Limitations:
Browser Use is incredible—but not magical, unfortunately… If you try to run your browser-based handler AI agent now, it’ll probably get blocked. That may occur because of a Google reCAPTCHA. If it somehow bypasses that, there’s still the Indeed human verification page powered by Cloudflare. These failures are especially common if you run the script on a server or in headless mode—which, let’s be honest, is exactly what you want. No one wants a machine tied up for minutes while it handles a task!
So yeah, all this sets up building an AI agent that fails… just like all the others. Was that a waste of time? Nope, as the tutorial isn’t over yet!
There’s still the most important step. The one that actually makes this whole thing work.
Integrate Agent Browser:
Your agent fails because the sites it interacts with can detect it as an automated bot. How does that happen? Tons of reasons, including:
- Browser fingerprinting: The browser session created by default in Playwright is super generic and doesn’t look like a real user.
- Rate limiters: Your agent ends up making too many requests in a short time (classic for automation, not humans), which triggers suspicion instantly.
- IP reputation: The more automation scripts you run from your IP, the more solutions like Cloudflare flag you as a potential bot—increasing the chances of a CAPTCHA or other verification.
So, what’s the solution? A browser that:
- Runs human-like sessions, mimicking real user behavior.
- Can solve CAPTCHAs automatically if they appear.
- Integrates with a proxy network with millions of rotating IPs to avoid rate limits.
- Runs in the cloud for infinite scalability.
- Integrates seamlessly with AI.
Is this a dream? Nope! It exists, and it’s called Agent Browser (aka Browser API)!
Follow the official Agent Browser integration guide, and you’ll end up on a page similar to the one shown in the source. Copy your connection URL (highlighted in red in the original resource) and add it to your .env file like so:
BRIGHT_DATA_BROWSER_AGENT_URL=<YOUR_AGENT_BROWSER_URL>Then, read it in agent.py and define the Browser object to instruct Browser Use to connect to the remote browser:
import os
from browser_use import Browser BRIGHT_DATA_BROWSER_AGENT_URL = os.getenv("BRIGHT_DATA_BROWSER_AGENT_URL")
browser = Browser( cdp_url=BRIGHT_DATA_BROWSER_AGENT_URL
)Next, pass the browser object to your agent:
agent = Agent( llm=llm, task=task, browser=browser, # <--- This is the crucial line
)Your AI agent will now execute tasks in remote Agent Browser instances, while no longer being blocked or interrupted. What a clutch!
Put It All Together:
Your final agent.py should contain:
from browser_use import ChatGoogle, Agent, Browser
from dotenv import load_dotenv
import json
import os # Read the environment variables from the .env file
load_dotenv() # The LLM powering the AI agent
llm = ChatGoogle(model="gemini-2.5-flash") # The task the AI agent will do on your behalf
task = """
Search on Google for software engineer jobs in New York.
1. Choose a job posting page.
2. On the chosen site, filter for jobs published within the last 24 hours.
3. For each job listing, extract the key details, including the job posting URL and the apply URL (if available).
4. Return all results as a JSON list.
""" # Read the Bright Data Browser Agent CDP URL from the env
BRIGHT_DATA_BROWSER_AGENT_URL = os.getenv("BRIGHT_DATA_BROWSER_AGENT_URL")
# Configure a remote browser
browser = Browser( cdp_url=BRIGHT_DATA_BROWSER_AGENT_URL
) # Define an AI agent to perform the task in the configured browser
agent = Agent( llm=llm, task=task, browser=browser,
) # Execute the AI agent
history = agent.run_sync() # Export the found jobs to a JSON output file
output_data = history.structured_output
with open("jobs.json", "w", encoding="utf-8") as f: json.dump(output_data, f, ensure_ascii=False, indent=4)Test it by running it with:
python agent.pyAs you can see from the GIF execution you can generate from Browser Use (perfect for debugging), the AI agent can now access Google, then Indeed, and filter jobs using the required criteria (posted in the last 24 hours).
The result will be a jobs.json file in your project folder, containing all the job data extracted from Indeed, ready for you to apply for:
[ { "job_title": "Software Engineer", "company": "Twitch Interactive, Inc.", "location": "New York, NY", "salary": "$99,500 - $200,000 a year", "employment_type": "Full-time", "benefits": [ "Parental leave", "401(k)", "Health insurance", "Paid time off", "Employee discount", "Vision insurance" ], "apply_url": "https://www.indeed.com/rc/clk?jk=d57f1f5ae2ce39b2&bb=KSTlUgVEMf-eBJjV36L3azapF2zEi4bBvUN2hIAcYXrYbXRZ5eWSuITPoUpo_Z8dlLX2UOM82XGDxHt0-Ahisofl6e8m0YvqC6Hh37bUv4Ph18Wp4oM2lqjW0jgm6q24kmXmCEOn4ZCXxMbVvGx1Lw%3D%3D&xkcb=SoAR67M3sAK4p3SDqh0LbzkdCdPP&fccid=fe2d21eef233e94a&vjs=3" }, // other job postings omitted for brevity... { "job_title": "Fullstack .NET Developer, Analyst", "company": "MUFG Bank, Ltd.", "location": "Hybrid work in Jersey City, NJ 07302", "salary": "$87,000 - $123,000 a year", "employment_type": "Full-time", "benefits": [ "Tuition reimbursement", "Paid parental leave", "Parental leave", "Health insurance", "Retirement plan", "Paid holidays" ], "apply_url": "https://www.indeed.com/rc/clk?jk=88f53bba78bb73d9&bb=KSTlUgVEMf-eBJjV36L3a5W1vAjJi2KOYfFuFmAdZolzMxeST7LmPwBH3Nh_N5WyZz05vH6_vGPa9dHkj6jgfo9yTQnbXCmfxYezDirnxuSYqjnNthL3s5UtUFYUkLK_DbCh8F545E0wDidVKUnxVQ%3D%3D&xkcb=SoBM67M3sAK4p3SDqh0FbzkdCdPP&fccid=3b98171e4a0fd997&vjs=3" }
]Wow! In around 40 lines of code, you just built an AI agent that can automate virtually any browser task for you! If you want to level up, you can even integrate it with logic to read your CV and apply for positions automatically, as shown in the official Browser Use example on GitHub.
Thanks to Bright Data’s Agent Browser integration in Browser Use, you can now craft an unstoppable AI agent that handles all the boring tasks that drain your time and energy. The AI agent revolution is now!
Beyond Job Hunting: Automate Any Browser Task
The beauty of this framework lies in its adaptability. By simply modifying the task description, your AI agent can be repurposed to tackle a vast array of browser-based workflows. Here are just a few ideas for boring tasks this intelligent agent can now handle for you:
- Find and schedule flights: Let the AI search for flights, compare options, and even book tickets based on your preferences, navigating complex booking sites with ease.
- Extract weather data for multiple cities: Get real-time weather info for all the cities you’re traveling to, compiling forecasts from various sources so you’re always prepared.
- Schedule calls for you: Rely on Calendly or similar tools, and the AI will arrange meetings according to your availability and specific criteria, sending out invites and managing confirmations.
- Track Amazon product prices and buy at low: Monitor product prices across e-commerce platforms and automatically purchase items when they hit your target price, ensuring you never miss a deal.
- Collect news headlines: Gather and summarize the latest news from multiple sources across the web, providing you with a tailored daily briefing so you don’t miss anything important.
- Buy groceries for you: Provide a shopping list, and the AI will automatically navigate online grocery stores, select items, and place your order, saving you precious time.
Want more ideas? Discover other AI agent use cases and scenarios.
Final Thoughts
Now you know how to build an AI agent that tackles boring, repetitive, dull, and time-consuming browser tasks for you. This level of automation wouldn’t be possible without a sophisticated approach. Browser Use provides one of the coolest AI agent libraries out there, offering the intelligence to reason and act. However, the real game-changer is Bright Data’s Agent Browser, which gives your AI unstoppable, agent-ready cloud browser instances that overcome anti-bot measures and provide truly human-like web interaction.
At Bright Data, our mission is simple: make AI accessible for everyone, everywhere—even for automated users. This powerful combination of Browser Use and Agent Browser democratizes advanced web automation, putting the power of intelligent, resilient AI agents directly into your hands.
Until next time, stay bold, and keep building the future of AI with creativity.
Ready to transform your workflow? Start building your own unstoppable AI agent today and reclaim your time!
Frequently Asked Questions
Q1: Why do most AI agents fail to deliver on their promise?
A1: Most AI agents struggle with realistic web interaction, getting blocked by anti-bot measures, the unpredictable nature of LLMs, and a lack of common-sense reasoning required to navigate dynamic web environments.
Q2: What are the key tools needed to build an unstoppable AI agent?
A2: The tutorial highlights Browser Use for powerful browser automation and LLM integration, combined with Bright Data’s Agent Browser to provide human-like browsing capabilities, bypass anti-bot defenses, and ensure consistent execution.
Q3: Can the AI agent built in this tutorial be used for tasks other than job hunting?
A3: Yes, the framework is highly adaptable. By simply modifying the task description given to the agent, it can be repurposed to automate a wide range of browser-based workflows, such as finding flights, tracking product prices, or scheduling calls.
Q4: How does Bright Data’s Agent Browser help AI agents overcome web interaction challenges?
A4: Agent Browser provides human-like browser sessions, automatic CAPTCHA solving, integration with a vast proxy network for IP rotation, and cloud-based execution, effectively bypassing common anti-bot techniques and ensuring the agent remains unblocked.
Q5: What programming language and libraries are used in this tutorial?
A5: The tutorial uses Python (version 3.11 or higher) along with the browser-use library for AI agent creation and browser automation, python-dotenv for environment variable management, and json for output handling. It integrates with LLMs like Gemini and Bright Data’s Agent Browser.
