New Rewritten version! Class renamed as well. Download had incorrect folder name and file name, though the class itself is named correctly. Don't code late at night.
- fixed csql bug (Timbo)
- updated when exportFull is true and CActiveDataProvider has pagination set. It now properly turns it off.
- update headers bug/var ref error (lunatic77)
- fixed row exclude bug
- updated to fix runtime enclosure/delimiter being set in toCSV()
CSVExport will allow you to create a csv file with optional row callbacks. By default it will return a string of comma delimited items, but can also write a file if you need one.
By passing an Array, CDbCommand, CSqlDataProvider, or CActiveDataProvider instance this extension will create a csv string or file, taking into account also any paging settings.
For extremely large csv files, you should just use mysql's select into file instead, as it will be really fast (you just need correct permissions).
Requirements ¶
Yii 1.1 PHP 5.2+
Usage ¶
Delimiter and Enclosure defaults to php defaults of , and ". You can change this with set functions, or at the call toCSV(). Many set* functions provide a fluent interface as well.
Yii::import('ext.ECSVExport');
// for use with array of arrays
$data = array(
array('key1'=>'value1', 'key2'=>'value2')
....
)
$csv = new ECSVExport($data);
$output = $csv->toCSV(); // returns string by default
echo $output;
// gives you something like
key1,key2
value1,value2
// OR
$csv = new ECSVExport($provider);
$content = $csv->toCSV();
Yii::app()->getRequest()->sendFile($filename, $content, "text/csv", false);
exit();
Writing To File
Yii::import('ext.ECSVExport');
// for use with array of arrays
$data = array(
array('key1'=>'value1', 'key2'=>'value2')
....
)
$filename = 'somewriteablefile.csv';
$csv = new ECSVExport($data);
$csv->toCSV($filename); // returns string by default
echo file_get_contents($filename);
// gives you something like
key1,key2
value1,value2
Human Readable headers
$headers = array('header_to_change'=>'new value');
$csv->setHeaders($headers);
// or
$csv->setHeader($currentHeaderValue, $newHeaderValue);
Per Row Callback
// callback must be is_callable by php or exception is thrown
$csv->setCallback(function($row){
$new = array();
foreach($row as $k=>$v) {
$new[$k] = $v * $v;
}
return $new;
});
Exclude Columns
$exclude = array('getlost','dontshow'...);
$csv->setExclude($exclude);
// or
$csv->setExclude($nameofcolumn);
Various Provider types
// CDbCommand
$cmd = Yii::app()->db->createCommand("SELECT * FROM track_test LIMIT 10");
$csv = new ECSVExport($cmd);
$csv->setOutputFile($outputFile);
$csv->toCSV();
// CSqlDataProvider
// Defaults to looping through all pages, use $csv->exportCurrentPageOnly(); to turn that off
$count=Yii::app()->db->createCommand('SELECT COUNT(*) FROM track_test WHERE campaign_id=1')->queryScalar();
$sql='SELECT * FROM track_test WHERE campaign_id=1';
$dataProvider=new CSqlDataProvider($sql, array(
'totalItemCount'=>$count,
'pagination'=>array(
'pageSize'=>10,
),
));
$csv = new ECSVExport($dataProvider);
$csv->exportCurrentPageOnly(); // if not set will loop through all pages!
$csv->setOutputFile($this->outputFile);
$csv->toCSV();
// CActiveDataProvider
$dataProvider = new CActiveDataProvider('Track', array(
'criteria'=>array(
'condition'=>'campaign_id=1 and type=1'
)
));
$csv = new ECSVExport($dataProvider);
$csv->setOutputFile($this->outputFile);
$csv->toCSV();
// Active Records
$csv = new ECSVExport(Track::model()->findAllByAttributes(array('campaign_id'=>1,'type'=>Track::TYPE_INT_SUCCESS)));
$csv->setOutputFile($this->outputFile);
$csv->toCSV();
When working with CActiveDataProviders, you can often run out of memory when working with large resultsets. $convertActiveDataProvider defaults to true, which creates a command out of provider. While this seems to fix memory problems, you are no longer able to use the with() function and will will to resort to joins. Sorry! Use dontConvertProvider() to turn that off.
Nice API + issues
API is really simple and nice. Moreover it allows to do a lot of things because uses data providers.
There are some minor assorted issues with code style:
Updated class
Thanks for your input. I've just updated the class according to your comment. The reason I choose the php://temp over php://memory is that php://temp stream operates in memory until it reaches a given size and then switches over to the file system (better explanation here: http://mikenaberezny.com/2008/10/17/php-temporary-streams/).
please help
Hi, thanks for your extension.
Could you please help me use it ?
what do you mean by "$csv->callback = function($row) {// do something with each row};" ?
do i need to put all the above code in an function or how should i do it ?
Closure
The callback can just be a closure, so you'll need to be running php 5.3 to do that, or it can just be something that can be called via call_user_func. Just pass one argument that will be the row.
Like $csv->callback = function($row){
$row['column'] = strtoupper($row['column']);
};
Return
$csv->callback = function($row){ $row['column'] = strtoupper($row['column']); return $row; };
can't get it work :(
i'm on php 5.2 and there is no update for my CentOS Linux ...
call_user_func
In the source the callback is just used like this:
$d = call_user_func($this->callback, $d);
take a look at the docs for call_user_func and you will see the options you have for assigning functions to that callback.
sorry for bothering you :(
first of all:
$provider = Produse::model()->createCSqlProvider(); //said :Produse does not have a method named "createCSqlProvider".
so i set it to:
i've read about call_user_func but o don't know what this extension expects from this. Could you please give me an example as you did for php 5.3?
CSqlDataProvider
Trying to clarify; the idea is that your model will have a function similar to how the search() method works in that the return value is a CSqlDataProvider. If you don't have one, no biggie, using findAll() works fine too, you just miss out on the features that CSqlDataProvider gives you.
call_user_func is pretty easy, it can be a function you define some place, or for example say you were wanting to use this in your controller:
class MyController extends CController { public function processRow($row) { $row['somecolumn'] = strtoupper($row['somecolumn']; return $row; } public function actionExportCSV() { // create $csv object etc ... // set the callback to use your public controller function. if you want. $csv->callback = array($this, 'processRow'); $csv->toCSV(); } }
progress
now it seems to work but i get this:
file_put_contents(../myfilename.csv) [<a href='function.file-put-contents'>function.file-put-contents</a>]: failed to open stream: Permission denied
could be from this ?
$fp = fopen("php://temp/maxmemory:{$this->memoryLimit}", 'w');
I'm on OS X and my server is on CentOS
permissions
no for this error
means you don't have write permissions to write that file in the toCSV call, and most likely that folder. If you are on OSX, then you can just do a cmd+i on that folder to see what the permissions are, or you can create that file then just chmod 777 on that file (though thats super liberal permissions, it will work).
one last thing
the saved file is perfect but the downloaded one via browser is not the content itself, is a number, the filesize of the saved .csv file.
$filename = Yii::app()->basePath.'/downloads/produse.csv'; $content = $csv->toCSV($filename, ",", "\""); Yii::app()->getRequest()->sendFile($filename, $content, "text/csv", false);
Thank you for everything.
Exit
Make sure you call exit after the sendFile. If you are not using the web log router, you don't have to use the last argument that is set to false. If your are using the weblog, you'll have to call exit or the log output will interfere with the download file. Try to just echo $csv->toCSV() without the sendFile and make sure the content is correct, if so the the problem lies in your implementation or sendFile use.
solved
$content = $csv->toCSV(); //print_r($content); -> PROPER OUTPUT HERE $filename = Yii::app()->basePath.'/downloads/produse.csv'; $content = $csv->toCSV($filename, ",", "\""); //print_r($content); -> WRONG OUTPUT (FILE SIZE INSTEAD OF CONTENT) Yii::app()->getRequest()->sendFile(basename($filename), @file_get_contents($filename), "text/csv"); //-> SOLUTION THAT WORKS exit();
returned file size
this line :
$content = $csv->toCSV($filename, ",", "\""); //print_r($content); -> WRONG OUTPUT (FILE SIZE INSTEAD OF CONTENT)
Look at the source. For one you don't need to pass , and " as arguments, those are default. If you are writing a file, it returns file size, not the content of the file. It returns the output of function file_put_contents. If you are not writing a file it returns the output of stream_get_contents. I'll change the doc to make that clearer.
use labels
How can I use the attributeLabels instead of the attributeId's for the header row?
headers
like this:
$csv->headers = array('first_name'=>'First Name');
can just be any function that returns an array, like the following:
it just has to be mapped like csv_header_name => human readable label
My preferred usage with ActiveRecord
You don't have to use an CSQLDataProvider, CActiveDataProvider is also supported according to the source code (can this be documented?)
I use it like this:
$provider = Donator::model()->search(); $csv = new CSVExport($provider); $csv->headers = Donator::model()->attributeLabels(); $content = $csv->toCSV(null, "\t", '"'); Yii::app()->getRequest()->sendFile('export.csv', $content, "text/csv", false); exit;
CActiveDataProvider usage
Yes, this is correct, bonus points for reading the source ;). I have had some odd memory errors related to attached behaviors on some active record result sets. So while using CActiveDataProvider is possible, depending on your result set you might get out of memory errors. Glad to see you are using it.
Re: CActiveDataProvider ignores criteria->with
Since internally a CActiveDataProvider is converted (maybe make this optional? how -crazy- is this overhead exactly?) to a direct sql command with createFindCommand(), the criteria->with part is ignored.
UPDATE: memory usage if we use CActiveDataProvider and retrieve array of AR objects:
createFindCommand(),
you have an code sample of how best to implement this? I'll take a look, but if you have something working that would be great
Re: Implement CActiveDataProvider
Take a look: http://pastebin.com/kD8NLBL7
I would suggest adding an option that defaults to building a command (much much faster).
Please document that createFindCommand() ignores the $criteria->with, and that you can use $criteria->join instead.
are you using .4
the .4 version has this already, is this what you are asking?
if($this->dataProvider instanceof CActiveDataProvider) { $criteria = $this->dataProvider->getCriteria(); $model = $this->dataProvider->model; $criteria = $model->getCommandBuilder() ->createCriteria($criteria,array()); $command = $model->getCommandBuilder() ->createFindCommand($model->getTableSchema(), $criteria); #echo $command->getText(); #exit; $data = $command->queryAll(); #print_r($data);exit; } else { $data = $this->dataProvider->getData(); }
Re: are you using .4
Take a look at my pastebin again :)
To clarify: I know 0.4 converts the CActiveDataProvider to a CDbCommand, I just added an option in case you don't want this auto-convert to happen (for example, if you are using $criteria->with).
got it and updated
thanks for the tip, updated to 0.5 to include. I added this, and defaulted to true as I used this in production and depend currently on the way it works.
public $convertActiveDataProvider = true;
Out of Memory
First, thanks for the great extension!
I've got a problem where the results from a queryall is too large to deal with in one go (38k+ records). Would it be difficult to rejig the extension to do some kind of paging and perhaps write the file in stages?
38k records
are you pulling back active records or just from csql?
Out of Memory | 38k Records
I'm using:
$provider = Yii::app()->db->createCommand('SELECT * ....')->QueryAll();
rewrite
ok im working on a rewrite right now that will help you with large record sets, I just have to finish up the testing today and should have it this week. do you have some sample data sets I can use to test against?
new version
there is a new version in devel for anyone who is interested:
http://thebusypixel.com/ECSVExport.zip
I will post as official after I finish more tests, as this new version supports passing CDbcommands and activerecord resultsets.
Re: rewrite
Thumbs up for the rewrite!
Large recordsets are now being handled by passing in a CDbCommand rather than a CSqlDataProvider [i.e. a QueryAll()].
added new class
old class is still available to download, but I suggest upgrading to the new one, its alot faster, uses less memory, has some more options.
Bugreport and quick idea
Bug
In 0.5, Exlude is not implemented properly.
It correctly excludes the headers, but not the rows. Replace the incorrect code in _writeRow with this:
// remove excluded if(count($this->_exclude) > 0) { foreach($this->_exclude as $e) { if(array_key_exists($e, $row)) { unset($row[$e]); } } }
Additional idea
You could extend from CComponent so we can do $csv->headers = array(), $csv->exclude = array(), etc.
How to send csv output to browser
And please include something like this in your documentation:
(..) $content = $csv->toCSV(); Yii::app()->getRequest()->sendFile('export.csv', $content, "text/csv", false); exit; // note that calling exit does not process log files
thanks for bug fix
I fixed that bug, and added the file output documentation here on this page, it was in the header on the actual file, but I had removed it by accident when I updated this page, thanks for pointing that out.
As for extending from CComponent, other than being able to set public properties, I don't see what advantages there would be. If you can think of something let me know. The other problem I have with just extending from CComponent is that you can't really restrict what those public properties are set to, ie; the setCallback function, which makes sure that you are only setting a callable function. You could check later when it is called, but I find it easier to debug if it fails right when you try to set it.
Requirement PHP 5.3
Why exactly is PHP 5.3 required? Keeping the requirements as low as possible or similar to current Yii version seems pretty important to me.
For example, our production environment (and probably many others) still runs on 5.2.
fix coming
I have a fix coming, but until I can upload it (on my phone right now) I just changed the description. Should be up today once I test and verify it.
1-off bug and non-variable passed by reference error
Great plugin, thank you! I have noticed/fixed a couple of bugs:
Hope that helps.
fixed
added fix, thanks for bug report
Letters mixed up
Hi, thanks for the great work.
It took me a few minutes to work out why the extension didn't want to load. You switched the 'C' and 'S' in the filenames of the latest releases...easy to overlook.
sorry!
what a careless error on my part, fixed in the download
Help with relations
Hey guys, I really like this extension but how do I get related columns into my $csv variable from a CActiveDataProvider?
I am trying to apply code to the search function that saves the filtered CActiveDataProvider in SESSION then use that SESSION variable as the datasource for the CSV.
I see you say you cant use with() unless you call dontConvertProvider(), however that seems to turn off my select and still not use my with().
Any help would be greatly appreciated.
$criteria->with = array( 'category'=>array( 'select'=>'category', 'together'=>true, ), 'location'=>array( 'select'=>'zip_id, street1, unit', 'together'=>true ), 'location.zip'=>array( 'select'=>'city, state', 'together'=>true ), 'post'=>array( 'select'=>'usr_name, phone_num', 'together'=>true ), ); $criteria->select=array('submitted', 'status','ordered_by', 'comments', 'rep');
try model relations
just noticed a bug with implementing the code mentioned below, download new version and then this should work.
$csv = new ECSVExport($criteria); $csv->dontConvertProvider(); // whatever columns you want $csv->setModelRelations(array('subid'=>'name','campaign'=>'name')); // OR $subid->name, $subid->value, $subid->foo, $subid->bar etc. $csv->setModelRelations(array('subid'=>array('name','value','foo','bar')));
Add some magic
foreach ($this->includeAttributes as $attr) { $row[$attr] = $model->{$attr}; }
Maybe add this at line 58 and a public array $includeAttributes to support magic component properties. e.g. to call
$user->getDisplayName(); // $csv->includeAttributes = array('displayName');
StripNewLines
Hi,
thanks for the extension.
My csv files won't import corectly so I changed:
$value = str_replace("\r\n"," ", $value); //To $value = str_replace(array("\n", "\r")," ", $value); //in function lambdaFail();
How to FORCE using enclousure ?
We need to have IN EVERY FIELD used the closure ( ".." or '..' )
We tried this extension only to do this, because native fputcsv only uses enclosure if it 'think' it's needed.
How to force ?!
Bug: First page repeated
I have found a problem using a CSQLProvider with pagination: the first page gets repeated for the total number of pages.
To fix:
if($this->_dataProvider instanceof CSqlDataProvider) { if($this->exportFull) { (...) for($i=1; $i<=$totalPages; $i++) { (...) // Change... $_getData = $this->_dataProvider->getData(); // to $_getData = $this->_dataProvider->getData(true); (...) } } else { (...)
thx timbo
added fix in new download.
Hi,
Thanx for this extension
Problem: Row without some data
I have a probleb when exporting data from a tabel. If there are rows with null values, the final .csv file doesn't have the double comma ",," that permit to put further data in the right columns.
For example
COLUMN1 - COLUMN2 - COLUMN3 - COLUMN4 data1 - data2 - data3 - data4 data1 - data2 - NULL - data4
I have
I wanted
so excel can put data in the right columns
code
Yii::import('ext.ECSVExport'); $dati = Yii::app()->db->createCommand("select ..."); $csv = new ECSVExport($dati); $output = $csv->toCSV(); $filename="file.csv"; Yii::app()->getRequest()->sendFile($filename, $output, "text/csv", false);
RESOLVED
With
$csv->convertActiveDataProvider=false;
Converting activedata provider to a cdbcommand causes the error and some problem with a filter I have implemented. Is slower but now it works.
Make excluded fields available to the callback
Use case:
Logic in the callback is dependent upon the value of a field that I don't want in the exported CSV. So I want to exclude the field but still have it for processing in the call back.
I amended my local code thus:
# This patch file was generated by NetBeans IDE # Following Index: paths are relative to: D:\www\vauxhall.partners\protected\extensions\ecsvExport # This patch can be applied using context Tools: Patch action on respective folder. # It uses platform neutral UTF-8 encoding and \n newlines. # Above lines and this line are ignored by the patching process. Index: ECSVExport.php --- ECSVExport.php Base (BASE) +++ ECSVExport.php Locally Modified (Based On LOCAL) @@ -579,6 +579,17 @@ if($row instanceof CActiveRecord) { $row = $row->getAttributes(); } + + if($this->stripNewLines) { + array_walk($row, array('ECSVExport','lambdaFail')); + } + + array_walk($row, array('ECSVExport','stripSlashes')); + + if(isset($this->_callback) && $this->_callback) { + $row = call_user_func($this->_callback, $row); + } + // remove excluded if(count($this->_exclude) > 0) { foreach($this->_exclude as $e) { @@ -588,17 +599,7 @@ } } - if($this->stripNewLines) { - array_walk($row, array('ECSVExport','lambdaFail')); - } - - array_walk($row, array('ECSVExport','stripSlashes')); - - if(isset($this->_callback) && $this->_callback) { - fputcsv($this->_filePointer, call_user_func($this->_callback, $row), $this->_delimiter, $this->_enclosure); - } else { fputcsv($this->_filePointer, $row, $this->_delimiter, $this->_enclosure); - } unset($row); }
support
hey I'm not in a position to support this extension anymore, if someone wants to take it over that would be great. :)
want to add new lines above headers in excel sheet
If i want to add two or three rows above the headers with some data how can i do that?
Resolved problem on null values in relations
Hi,
I have a problem exporting with null values in correct column
(check comment #c15025).
I have resolved with
$csv->convertActiveDataProvider=false;
but with attributes of related tables the problem still remains.
So I have modified the source of the extension at line 447, adding a comment before the two if
if(isset($model->$relation->$subvalue) && $model->$relation->$subvalue)
because I think this would skip null values of attributes and will cause shifting of datas.
The snippet of code is:
foreach($this->_modelRelations as $relation=>$value) { if(is_array($value)) { foreach($value as $subvalue) { //if(isset($model->$relation->$subvalue) && $model->$relation->$subvalue) $row[$relation.'['.$subvalue.']'] = $model->$relation->$subvalue; } } else { //if(isset($model->$relation->$value) && $model->$relation->$value) $row[$relation.'['.$value.']'] = $model->$relation->$value; } }
Extra Row is adding above header using data array
Hi,
I need a help for extra row is being added on above of header. I am using an array to pass in plug in and writing header and data. But extra blank row is adding before header.
Can you please help me?
If you have any questions, please ask in the forum instead.
Signup or Login in order to comment.