Skip to content

Burr Integration throws errors with SmartScraperGraph example #975

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
flst01 opened this issue May 8, 2025 · 1 comment
Open

Burr Integration throws errors with SmartScraperGraph example #975

flst01 opened this issue May 8, 2025 · 1 comment
Labels
bug Something isn't working

Comments

@flst01
Copy link

flst01 commented May 8, 2025

Describe the bug
When using the example (https://scrapegraph-ai.readthedocs.io/en/latest/getting_started/examples.html) with Burr integration (https://scrapegraph-ai.readthedocs.io/en/latest/scrapers/graph_config.html#burr-integration) Fetch throws an error. After fixing the error in Fetch, ParseNode throws an error.

To Reproduce
Steps to reproduce the behavior:
After starting burr in terminal by using "burr", run the following:
"import os
from dotenv import load_dotenv
from scrapegraphai.graphs import SmartScraperGraph
from scrapegraphai.utils import prettify_exec_info

load_dotenv()
openai_key = os.getenv("OPENAI_API_KEY")

graph_config = {
"llm": {
"api_key": openai_key,
"model": "openai/gpt-4.1-nano",
},
"burr_kwargs": {
"project_name": "test-scraper",
"app_instance_id": 1,
}
}

smart_scraper_graph = SmartScraperGraph(
prompt="List me all the projects with their description.",
source="https://perinim.github.io/projects/",
config=graph_config
)

result = smart_scraper_graph.run()"

Expected behavior
Burr integration should work without error

Acutal Behaviour
Error in Fetch: ValueError: Action ParseNode attempted to write to keys {'content'} that it did not declare. It declared: (['parsed_doc'])!
Error in ParseNode: ValueError: Action ParseNode attempted to write to keys {'content'} that it did not declare. It declared: (['parsed_doc'])!

Desktop :

  • OS: macOS Sequoia 15.4.1

Additional context
Solved by fork. See Pull Request.
Might still affect other Graph types, havent checked...

Copy link

dosubot bot commented May 8, 2025

Hello @flst01! I'm Dosu, your friendly bug-solving bot here to help you tackle issues, answer questions, and navigate the path to contribution. While you wait for a human maintainer, feel free to let me know how I can assist you! 🚀

The errors you're encountering with the Burr integration in the SmartScraperGraph example are due to a mismatch in the expected output keys for the Fetch and ParseNode actions. Specifically, the error messages indicate that these actions are attempting to write to the key 'content', which is not declared in their configuration.

For the FetchNode action, the expected output key should be 'doc', and for the ParseNode action, it should be 'chunks' [1][2]. The error suggests that the graph configuration or the Burr integration setup might be incorrectly specifying or expecting the 'content' key instead of the correct ones.

To resolve this issue, ensure that the graph configuration correctly declares the output keys for these actions. You might need to adjust the configuration to match the expected keys or review the fork that resolved the issue for specific changes made to address this mismatch.

To continue talking to Dosu, mention @dosu.


Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other

@dosubot dosubot bot added the bug Something isn't working label May 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant