The Crawlomatic Multisite Scraper Post Generator is a powerful WordPress plugin that facilitates content scraping and post generation, offering a wide array of features. Here’s an overview of its capabilities:
Key Features Overview:
- Webpage Crawling & Scraping: The plugin crawls given seed URLs, extracting content from each crawled URL. It allows customizable crawling settings like depth, rate, maximum articles, targeting specific links, and more.
- Live Scraper Shortcode: The v2.0 update introduced a live scraper shortcode ([crawlomatic-scraper]), allowing real-time data extraction for posts, pages, or sidebars. It includes caching for optimized performance.
- Multiple Scraping Techniques:
- CSS Selector, XPath, or Regex-based content querying.
- Execution of JavaScript on scraped HTML.
- Dynamic conversion of scraped content to different character encodings.
- Scraping RSS feeds for links and articles.
- Paginated crawling and importing support.
- Integration & Compatibility:
- Supports WooCommerce product scraping, including variants.
- Integration with Google and Bing to scrape search engine results for custom keywords.
- Capability to scrape .onion websites from the Dark Web using Tor Browser and Puppeteer.
- Media & Visual Elements:
- Screenshots of crawled pages can be used in generated posts’ content.
- Embeds videos from various platforms using crawling and scraping methods.
- Post Customization & Handling:
- Customizable post status, category, tags, and other metadata.
- Content customization using various shortcodes and tools like a keyword replacer, random sentence generator, etc.
- Scheduled rule runs for automatic post generation.
- Control & Management:
- Respect for robots.txt and robots HTML headers.
- Various constraints and settings like post length limitations, keyword checking, etc.
- Logging of plugin activities for better oversight.
- Integration & Export:
- Integration with Zapier and webhook support for custom service integration.
- Ability to save and restore plugin rule list from files for easier management.
- Miscellaneous:
- Option to delete generated posts after a specific period.
- Detailed logging of plugin activities.
- Custom field and taxonomy support for generated posts.
- Ability to copy images locally or embed them remotely.
This plugin empowers users to extract content from various websites, allowing for customization, automation, and extensive scraping and post generation options, making it a versatile tool for content aggregation and publication within WordPress.
