PHP Classes

Content downloader: Download content from many sites at the same time

Recommend this page to a friend!
  Info   View files Example   View files View files (2)   DownloadInstall with Composer Download .zip   Reputation   Support forum (1)   Blog    
Ratings Unique User Downloads Download Rankings
Not enough user ratingsTotal: 538 All time: 5,585 This week: 148Up
Version License PHP version Categories
content-downloader 0.1GNU General Publi...5.0PHP 5, Web services


This class can download content from multiple sites at the same time.

It can add to a queue one or more URLs of remote sites from which it is necessary to download content.

The class uses the Curl multiple functions to send the requests to all remote sites at once and returns the downloaded content or the error messages in case it was not possible to download the content from any of the sites.

Picture of magog
  Performance   Level  
Name: magog <contact>
Classes: 3 packages by
Country: Russian Federation Russian Federation
Age: 47
All time rank: 285878 in Russian Federation Russian Federation
Week rank: 344 Up15 in Russian Federation Russian Federation Up
Innovation award
Innovation award
Nominee: 2x



/*... some code ...*/

include_once 'CContentDownloader.php';

// Create CContentDownloader instance
$contRecivier = new CContentDownloader();

// Get contents of all pages in some array
foreach($pages as $k => $page){
$uri = $page['host'].$page['uri'];
// Add source & remember index in stack
$pages[$k]['uri_id'] = $contRecivier->addSource($uri);

// Get contents of requested url`s
$contents = $contRecivier->getContents();

// Proccess contents
    foreach (
$pages as $k => $page){
$contents[$page['uri_id']]) &&
is_array($contents[$page['uri_id']]) &&
$contents[$page['uri_id']]['error'] === false

$content = $contents[$page['uri_id']];
// Process curl info array (headers)
if($content['headers']['http_code'] != 200){
                throw new
Exception("Incorrect HTTP code ({$content['headers']['http_code']}).");
strpos( $content['headers']['content_type'], 'text/html' ) === FALSE){
                throw new
Exception("Unsupported format ({$content['headers']['content_type']}).");
/* ... do something with content */
echo $content['content'];
        } else {
/* proccess curl error */
throw new Exception("Curl error #({$contents[$page['uri_id']]['error']}).");

/*... some code ...*/


  Files folder image Files  
File Role Description
Plain text file CContentDownloader.php Class Content Downloader class
Accessible without login Plain text file example.php Example How to use

 Version Control Unique User Downloads Download Rankings  
This week:0
All time:5,585
This week:148Up