Search for autonomous research agents

Give your agent a primitive it can call dozens of times per task.

For

Builders of autonomous research agents — deep research products, OSINT tools, AI analysts — where the LLM plans its own search sequence to answer multi-hop questions.

The pain

Without a real retrieval layer.

Generic search APIs are tuned for end-user SERPs, not for an agent that wants to fan out across 30 sub-queries, reconcile contradictions, and write a 5-page report. Latency, structured shape, and credit cost dominate the experience.

With SiftQ

One call, the right shape.

SiftQ is built for the agent retrieval pattern: predictable per-query latency, scored results, scope-aware response shapes (web / scholar / news / podcast / video / image), and per-query pricing that maps cleanly onto agent budgets.

Recommended config

Start with this shape.

scopemixed — webpage for breadth, scholar for depth, news for recency
size10-20
optionsincludeSummary=false (let your agent reason over raw snippets) · Multiple scopes per task
Working code

Paste, run, ship.

Python — Multi-hop research agent loop
import os, json, anthropic, requests

claude = anthropic.Anthropic()
HEADERS = {"Authorization": f"Bearer {os.environ['SIFTQ_API_KEY']}"}

def siftq(q: str, scope: str = "webpage", size: int = 10) -> list[dict]:
    r = requests.post(
        "https://api.siftq.com/v1/search",
        headers=HEADERS,
        json={"q": q, "scope": scope, "size": str(size)},
        timeout=15,
    )
    r.raise_for_status()
    data = r.json()
    return data.get("webpages") or data.get("scholars") or data.get("podcasts") or []

TOOLS = [{
    "name": "search",
    "description": "Search the web. Use scope=scholar for papers, news for recency.",
    "input_schema": {
        "type": "object",
        "required": ["q"],
        "properties": {
            "q": {"type": "string"},
            "scope": {"type": "string", "enum": ["webpage", "news", "scholar"]},
        },
    },
}]

def research(topic: str, max_steps: int = 8) -> str:
    messages = [{"role": "user", "content": f"Research: {topic}"}]
    for _ in range(max_steps):
        resp = claude.messages.create(
            model="claude-sonnet-4-6", max_tokens=2048,
            tools=TOOLS, messages=messages,
        )
        if resp.stop_reason == "end_turn":
            return resp.content[0].text
        for block in resp.content:
            if block.type == "tool_use":
                hits = siftq(block.input["q"], block.input.get("scope", "webpage"))
                messages.extend([
                    {"role": "assistant", "content": resp.content},
                    {"role": "user", "content": [{"type": "tool_result",
                        "tool_use_id": block.id,
                        "content": json.dumps(hits[:5])}]},
                ])
    return "Step budget exhausted."

print(research("How has GPU pricing evolved in 2024-2025?"))
Why SiftQ for this
FAQ

Likely questions.

How many searches will a typical agent task make?

Deep research agents commonly fire 15-40 searches per task — initial breadth scan, follow-ups, cross-references, verification. Plan your free-tier or paid budget accordingly.

Should the agent pick the scope, or should I hard-code it?

Let the agent pick. Expose scope as a tool parameter with a brief description; modern models (Claude 4.x, GPT-5) reliably pick scholar for academic questions, news for "what happened", webpage for everything else.

How do I stop runaway tool calls?

Set a hard step budget (e.g. 12 search calls max) and a hard cost cap per task. If your agent hits the cap, fail fast and ask the user for narrower direction.

Keep reading
Use case
Web search for RAG applications
Ground your LLM in the live web — not a stale training cutoff.
Use case
Competitive intelligence on demand
Monitor competitors, markets, and signals — programmatically.
Glossary
AI agent
An LLM-driven system that takes actions in the world, often by calling tools and APIs in a loop.
Glossary
Tool use
An LLM's ability to invoke external functions / APIs as part of generating its response.
Glossary
Long-tail retrieval
Returning relevant results for rare, specific, or obscure queries — not just the popular ones.
Glossary
MCP (Model Context Protocol)
Anthropic-proposed open protocol for connecting LLMs to external tools and data sources.

Try it on your
real workload.

The free tier covers most prototypes. No credit card.