user9291211
user9291211

Reputation: 1

crawl4ai gives Error: 'NoneType' object has no attribute 'new_context'

I am trying to scrape data from www.example.com but the below code returns error :

import asyncio
from crawl4ai import AsyncWebCrawler
from crawl4ai.async_configs import BrowserConfig, CrawlerRunConfig


async def main() :
    try :
    # Create an instance of AsyncWebCrawler
        # async with AsyncWebCrawler as crawler :
        crawler = AsyncWebCrawler()
            # Run the crawler on a URL
        result = await crawler.arun(url="https://crawl4ai.com", config=CrawlerRunConfig())
            # Print the extracted content
        if result.markdown():
                print(result.markdown())
        else:
                print("No Markdown content found.")
    except Exception as e:
        print(f"An error occurred: {e}")

# Run the async main function
if __name__ == "__main__" :
        try :
            asyncio.get_running_loop()
            loop = asyncio.ProactorEventLoop()
            asyncio.set_event_loop(loop)
            asyncio.run(main())
            print("in try")
        except RuntimeError :
            print("in except")
            asyncio.run(main())

Error as below :

× Unexpected error in _crawl_web at line 664 in create_browser_context (..\mahakumbh_venv\lib\site-                   │
│ packages\crawl4ai\async_crawler_strategy.py):                                                                         │
│   Error: 'NoneType' object has no attribute 'new_context'                                                             │
│                                                                                                                       │
│   Code context:                                                                                                       │
│   659               }                                                                                                 │
│   660               # Update context settings with text mode settings                                                 │
│   661               context_settings.update(text_mode_settings)                                                       │
│   662                                                                                                                 │
│   663           # Create and return the context with all settings                                                     │
│   664 →         context = await self.browser.new_context(**context_settings)                                          │
│   665                                                                                                                 │
│   666           # Apply text mode settings if enabled                                                                 │
│   667           if self.config.text_mode:                                                                             │
│   668               # Create and apply route patterns for each extension                                              │
│   669               for ext in blocked_extensions:                                                                    │
└───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

Crawl failed: Unexpected error in _crawl_web at line 664 in create_browser_context (..\mahakumbh_venv\lib\site-packages\crawl4ai\async_crawler_strategy.py):
Error: 'NoneType' object has no attribute 'new_context'

Code context:
 659               }
...
 667           if self.config.text_mode:
 668               # Create and apply route patterns for each extension
 669               for ext in blocked_extensions:
in try

I am expecting to print the scraped data from the url. I have tried asyncio.arun(main()) instead of asyncio.run(main()) but it returns with Error: 'NoneType' object has no attribute 'new_context

Upvotes: 0

Views: 39

Answers (0)

Related Questions