Skip to content

support delimiter and encode #109

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
do9iigane wants to merge 2 commits into
base: master
Choose a base branch
from
Open

support delimiter and encode #109

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e9c8b83
tsv extension support
bil-bot Jul 15, 2014
b4a8329
support delimiter and encode
do9iigane Jul 15, 2014
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 26 additions & 17 deletions Model/Datasource/CsvSource.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@
* 'extension' => 'csv', // File extension
* 'readonly' => true, // Mark for read only access
* 'recursive' => false // Only false is supported at the moment
* 'delimiter' => ',', // File delimiter
* 'encoding' => 'UTF8' // File convert to encode
* );
*/
App::uses('DataSource', 'Model/Datasource');
Expand Down Expand Up @@ -182,23 +184,23 @@ public function describe($model) {
* @return boolean True, Success
*/
protected function _getDescriptionFromFirstLine(Model $model) {
$filename = $model->table . '.' . $this->config['extension'];
$handle = fopen($this->config['path'] . DS . $filename, 'r');
$line = rtrim(fgets($handle));
$dataComma = explode(',', $line);
$dataSemicolon = explode(';', $line);

if (count($dataComma) > count($dataSemicolon)) {
$this->delimiter = ',';
$this->fields = $dataComma;
$this->maxCol = count($dataComma);
} else {
$this->delimiter = ';';
$this->fields = $dataSemicolon;
$this->maxCol = count($dataSemicolon);
}
fclose($handle);
return true;
$filename = $model->table . '.' . $this->config['extension'];
$handle = fopen($this->config['path'] . DS . $filename, 'r');
$line = rtrim(fgets($handle));

$this->delimiter = isset($this->config['delimiter']) ? $this->config['delimiter'] : $this->delimiter;

$data = explode($this->delimiter, $line);

if (isset($this->config['encoding'])) {
mb_convert_variables($this->config['encoding'], "ASCII,UTF-8,SJIS-win", $data);
Copy link
Member

@lorenzo lorenzo Jul 15, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is this needed for?

Copy link
Author

@do9iigane do9iigane Jul 15, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mb_convert_variables is not work for multi byte keys
ex)
1row | タイトル(title)|
2row | いいい|
3row | ううう|
↓
array (
0 =>
array (
'modelname' =>
array (
'"{xxxxxx:bad encode}(title)"' => 'いいい',

}
$this->fields = $data;
$this->maxCol = count($data);

fclose($handle);
return true;

}

/**
Expand Down Expand Up @@ -313,6 +315,13 @@ public function read(Model $model, $queryData = array(), $recursive = null) {
if ($model->findQueryType === 'count') {
return array(array(array('count' => count($resultSet))));
}

if (isset($this->config['encoding'])) {
Copy link
Member

@lorenzo lorenzo Jul 15, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indentation level in this block is incorrect

mb_convert_variables($this->config['encoding'], "ASCII,UTF-8,SJIS-win", $resultSet);

return $resultSet;
}

return $resultSet;
}

Expand Down