Giter VIP home page Giter VIP logo

objects-to-csv's Introduction

Convert array of objects into a CSV file

Converts an array of JavaScript objects into the CSV format. You can save the CSV to file or return it as a string.

The keys in the first object of the array will be used as column names.

Any special characters in the values (such as commas) will be properly escaped.

Usage

const ObjectsToCsv = require('objects-to-csv');

// Sample data - two columns, three rows:
const data = [
  {code: 'CA', name: 'California'},
  {code: 'TX', name: 'Texas'},
  {code: 'NY', name: 'New York'},
];

// If you use "await", code must be inside an asynchronous function:
(async () => {
  const csv = new ObjectsToCsv(data);

  // Save to file:
  await csv.toDisk('./test.csv');

  // Return the CSV file as string:
  console.log(await csv.toString());
})();

Methods

There are two methods, toDisk(filename) and toString().

async toDisk(filename, options)

Converts the data and saves the CSV file to disk. The filename must include the path as well.

The options is an optional parameter which is an object that contains the settings. Supported options:

  • append - whether to append to the file. Default is false (overwrite the file). Set to true to append. Column names will be added only once at the beginning of the file. If the file does not exist, it will be created.
  • bom - whether to add the Unicode Byte Order Mark at the beginning of the file. Default is false; set to true to be able to view Unicode in Excel properly. Otherwise Excel will display Unicode incorrectly.
  • allColumns - whether to check all array items for keys to convert to columns rather than only the first. This will sort the columns alphabetically. Default is false; set to true to check all items for potential column names.
const ObjectsToCsv = require('objects-to-csv');
const sampleData = [{ id: 1, text: 'this is a test' }];

// Run asynchronously, without awaiting:
new ObjectsToCsv(sampleData).toDisk('./test.csv');

// Alternatively, you can append to the existing file:
new ObjectsToCsv(sampleData).toDisk('./test.csv', { append: true });

// `allColumns: true` collects column names from all objects in the array,
// instead of only using the first one. In this case the CSV file will
// contain three columns:
const mixedData = [
  { id: 1, name: 'California' },
  { id: 2, description: 'A long description.' },
];
new ObjectsToCsv(mixedData).toDisk('./test.csv', { allColumns: true });

async toString(header = true, allColumns = false)

Returns the CSV file as a string.

Two optional parameters are available:

  • header controls whether the column names will be returned as the first row of the file. Default is true. Set it to false to get only the data rows, without the column names.
  • allColumns controls whether to check every item for potential keys to process, rather than only the first item; this will sort the columns alphabetically by key name. Default is false. Set it to true to process keys that may not be present in the first object of the array.
const ObjectsToCsv = require('objects-to-csv');
const sampleData = [{ id: 1, text: 'this is a test' }];

async function printCsv(data) {
  console.log(
    await new ObjectsToCsv(data).toString()
  );
}

printCsv(sampleData);

Requirements

Use Node.js version 8 or above.

objects-to-csv's People

Contributors

anton-bot avatar dependabot[bot] avatar dygufa avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

objects-to-csv's Issues

Adding double quotes to the header as well as data

Could you please let me know if there an option to add double quotes header as well as data before adding to csv?

I was able to add it to data and header but I don't find any option to disable header while call toDisk method.

Thanks in advance

semicolon

Is it possibile to use the semicolon for the cdv, instead of the commas?

Any plan to add a file appender

Hi,

With the current implementation, every time you write to the same file the previous file content will be overridden. Is there any plan to add a file append? I took a quick look at the code and I think fs.appendFileSync() would work as pretended, along with adding this as an optional argument.

Very handy dependency btw
Cheers ๐Ÿ˜„

Automatically convert boolean to INT in the CSV

When I try to create a CSV with below data, CSV file was created with value 1 for true and empty for false but I expect to be updated as true or false

[
  {
    email_address: '[email protected]',
    domain: 'gmail.com',
    valid_syntax: true,
    disposable: false,
    webmail: true,
    deliverable: true,
    catch_all: false,
    gibberish: false,
    spam: false
  },
  {
    email_address: '[email protected]',
    domain: 'gmail.com',
    valid_syntax: true,
    disposable: false,
    webmail: true,
    deliverable: true,
    catch_all: false,
    gibberish: false,
    spam: false
  },
  {
    email_address: '[email protected]',
    domain: 'gmail.com',
    valid_syntax: true,
    disposable: false,
    webmail: true,
    deliverable: true,
    catch_all: false,
    gibberish: false,
    spam: false
  }
]

Screenshot 2022-01-26 at 2 15 46 PM

saving csv to parent directories not working Windows 10

I'm on windows 10, node v 10.15.3

I'm using csv-parser to read through a csv and alter the data row-by-row in an fs.createReadStream() call.
After the read, I try to use objects-to-csv to save the altered array of objects as new csv file. this works if I use a filename like './file.csv'. But, when I try to save in parent directories, the file is created but empty, and 'error caught' is thrown for every row of csv that is parsed.

What am I doing wrong? I have tried many path.parse/path.resolve/etc to troubleshoot but I still cannot figure it out.

const file = "C:/path/to/my/file.csv"
const csv = require('csv-parser')
const ObjectsToCsv = require('objects-to-csv')
absfilepath = path.resolve(file)

var csvarray = [];
fs.createReadStream(absfilepath)
.pipe(csv())
.on('data', function(data){
try {
csvarray.push(cleanData(data))
}
catch(err) {
console.log('error caught')
}
})
.on('end', function(data) {
// new ObjectsToCsv(csvarray).toDisk('./file.csv'); this works
new ObjectsToCsv(csvarray).toDisk('../dirname/subdirname/file.csv'); // throws error on 'data' file created but empty
})
})

Special characters are not supported

When I write a CSV file with characters liek the native danish ร†, ร˜ and ร… its not encoded correctly. Is it possible that you can add an option to encode at UTF-8?

`.toDisk` adds another row of columns to the output file.

For example: The input file looks like this:

name,status,url
A,B,C

After reading this as an array of objects, it becomes this:

[
  {
    name: A,
    status: B,
    url: C
  }
]

But after using .toDisk it becomes:

name,status,url
name,status,url
A,B,C

Adding column names to data every other line, stops after 38 rows (line 76 is the last one with column names)

I'm running this on 100 files currently, pending a run on over 14k files. When I tested from a local directory with 3 files this didn't happen, but now I'm testing on 100 files from our network I'm getting the column names added as data in the columns every other line. Line 1 is the column names, line 2 is the first row of data, line 3 is the column names, line 4 is the next row of data, etc. Line 76 of the CSV is the last spot where the column names appear. Currently not throwing any errors relating to this.

Sorting of Columns should be optional when allColumns is true in the toString() method

There is a scenario where the order of the columns matter and there can be dynamic columns added at the end of the object. So, it is required to have the allColumns paramater set to true, but then the colums are sorted by default. This should be moved to the parameter of the toString() method and set to true by default when allColumns is true.

Only keys in the first item are added as columns.

Only keys in the first item are added as columns. Sometimes, I may have data as, to use a modified README.md example:

[
{code: 'HK', name: 'Hong Kong'},
{code: 'KLN', name: 'Kowloon', aNewColumn:'test'},
{code: 'NT', name: 'New Territories'},
]

This extra key in the second object is not currently processed. To keep things predictable, it would also need to be sorted in some way.

Is it possible to save data into multiple sheets?

I'm trying to spread my data in multiple sheets in CSV that I can switch in between, is that possible?

To be clear, I have 3 data arrays that I want to fit into one CSV file, they all have different data and should not be together

Update: After researching a bit I've noticed that CSV doesn't allow recording data in multiple sheets? But maybe it's possible to record data at least in separate columns that don't release to each other? For example: Data A then space then Data B

Column Shift due to certain undefined values in Array of objects

Using version : "objects-to-csv": "^1.3.6",

During a bulk upload/writing a CSV file,
noticed a column shift due to a few undefined values in the array of objects.
On writing a file with one set of data at a time, noticed that the column itself is not created if the value of the object's key for that column is 'undefined'.

Handled that at the array level,
Should be some kind of pointer or value to be passed as 'undefined string'.
Since in some cases it's not traceable which values can be undefined for example in scraped data set of a web site.

Maximum call stack size exceeded with a lot of data

When data is big (I had 432504 objects), this error happens:
RangeError: Maximum call stack size exceeded
at convert (/project/node_modules/objects-to-csv/index.js:134:12)
at ObjectsToCsv.toString (/project/node_modules/objects-to-csv/index.js:96:18)
at ObjectsToCsv.toDisk (/project/node_modules/objects-to-csv/index.js:56:27)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.