Giter VIP home page Giter VIP logo

linez's Introduction

linez

Parses lines from text, preserving line numbers, offsets and line endings.

Build Status Dependency Status npm version codecov

npm

Getting Started

Installation

$ npm install linez

Importing

TypeScript

import * as linez from 'linez';

Babel/ES6+

import linez from 'linez';

JavaScript

var linez = require('linez');

Introduction

By default, linez uses /\r?\n/g as the regular expression to detect newline character sequences and split lines. This regular expression is tuned for performance and only covers the most common newline types (i.e., \n and \r\n). If you have need for more newline character sequences, you can configure linez with the configure method.

linez.configure({
  newlines: ['\n', '\r\n', '\r', '\u000B']
});

Setting this property will automatically create a piped regular expression for you and use it in any future linez() calls. You can make up your own newlines if you want. Linez doesn't care one way or the other.

linez.configure({
  newlines: ['foo', 'bar']
});

This would be converted into /(foo|bar)/g. Newlines are just strings. They can be anything. There are, however, some known newline character sequences. Should you need them, refer to the following table:

String Unicode Name
\n U+000A Line feed
\r\n U+000D, U+000A Carriage Return + Line Feed
\r U+000D Carriage Return
\u000B U+000B Vertical Tab
\u000C U+000C Form Feed
\u0085 U+0085 Next Line
\u2028 U+2028 Line Separator
\u2029 U+2029 Paragraph Separator

Byte Order Marks

Also referred to as BOM signatures, these are the bytes at the beginning of a file that indicating the encoding in which the file is written. Currently, linez only reads BOMs to detect the encoding and does not take into account the contents of the file.

Supported BOMs

  • utf-8-bom
  • utf-16le
  • utf-16be

Unsupported BOMs

  • utf-32le
  • utf-32be

If linez detects an unsupported BOM, an error will be thrown, indicating that decoding the detected charset is not supported.

Default decoding

By default, the document will attempt to be decoded as utf8. This is the default behavior of the Node API's conversion from buffers into strings.

API

configure(options: IOptions)

Configures linez to use the supplied options. Currently, only the newlines property is available, where you can specify any number of newline character sequences.

linez.configure({
  newlines: ['\n', '\r\n', '\r', '\u000B']
});

resetConfiguration()

Resets the configuration to the default settings, using /\r?\n/g as the newlines regular expression.

Document

constructor(public lines: Line[]);

Calling the toString() method converts the document's lines into a string, discarding information about line numbers and offsets.

Line

interface Line {
  offset: number;
  number: number;
  text: string;
  ending: string;
}

Options

interface Options {
  newlines?: string[];
}

linez(file: string|Buffer): Document

Parses text into a Document.

The specs show some great usage examples.

var lines = linez('foo\nbar\nbaz').lines;
lines[1].offset; // 4
lines[1].number; // 2
lines[1].text; // bar
lines[1].ending; // \n

Note: You can also pass-in a Buffer.

License

Released under the MIT license.

linez's People

Contributors

bitdeli-chef avatar denis-sokolov avatar jedmao avatar jednano avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

linez's Issues

Node v4+ NOT compatibility

$ node -v
v6.9.1
$ npm test

> [email protected] test /mnt/e/work/linez
> gulp test

[15:19:18] Using gulpfile /mnt/e/work/linez/gulpfile.js
[15:19:18] Starting 'clean'...
[15:19:18] Finished 'clean' after 177 μs
[15:19:18] Starting 'clean'...
[15:19:18] Starting 'tslint'...
[15:19:18] Finished 'tslint' after 578 ms
[15:19:18] Finished 'clean' after 593 ms
[15:19:18] Starting 'scripts'...
[15:19:20] Finished 'scripts' after 1.42 s
[15:19:20] Starting 'build'...
[15:19:20] Finished 'build' after 2 μs
[15:19:20] Starting 'test'...


  StringFinder
    constructor
      ✓ requires a string[] or Regex in the constructor
      ✓ requires regular expression to have the global flag
    findAll method
      ✓ finds 2 needles in a haystack

  linez
    ✓ parses empty text
    ✓ parses a single line with no line ending
    ✓ parses a single line with a line ending
    ✓ sets proper line offsets
    ✓ sets proper line numbers
    ✓ sets proper line text
    ✓ sets proper line endings
    ✓ supports resetting the configuration
    configure method
      ✓ supports a custom newlines string array
      ✓ errors when no configuration options are sent
    Document
      ✓ constructs a document with lines
      ✓ converts doc into a buffer with toBuffer() (66ms)
      ✓ converts a signed doc into a buffer with toBuffer()
      ✓ converts an unsigned doc into a string with toString()
      ✓ throws when trying to set an unsupported charset
      byte order marks
        ✓ detects and decodes a utf-8-bom document
        ✓ detects and decodes utf-16le bom document
        1) detects and decodes utf-16be bom document
        ✓ throws an unsupported charset error for utf-32le bom
        ✓ throws an unsupported charset error for utf-32be bom
        ✓ decodes unsigned docs as utf8 by default


  23 passing (122ms)
  1 failing

  1) linez Document byte order marks detects and decodes utf-16be bom document:

      AssertionError: expected '뢸숃\u0000' to equal 'foo'
      + expected - actual

      -뢸숃
      +foo

      at Context.<anonymous> (js/linez.spec.js:9:9230)



[15:19:20] 'test' errored after 566 ms
[15:19:20] Error in plugin 'gulp-mocha'
Message:
    1 test failed.
npm ERR! Test failed.  See above for more details.

Doesn’t work on Travis CI (node 0.10.x), throws Unhandled rejection TypeError

$ eclint check $(git ls-files)
Unhandled rejection TypeError: Object # h has no method 'equals'
    at detectCharset (/home/travis/.nvm/v0.10.36/lib/node_modules/eclint/node_modules/linez/js/linez.js:35:41)
    at linez (/home/travis/.nvm/v0.10.36/lib/node_modules/eclint/node_modules/linez/js/linez.js:19:19)
    at /home/travis/.nvm/v0.10.36/lib/node_modules/eclint/js/eclint.js:69:27
    at tryCatcher (/home/travis/.nvm/v0.10.36/lib/node_modules/eclint/node_modules/editorconfig/node_modules/bluebird/js/main/util.js:26:23)
    at Promise._settlePromiseFromHandler (/home/travis/.nvm/v0.10.36/lib/node_modules/eclint/node_modules/editorconfig/node_modules/bluebird/js/main/promise.js:503:31)
    at Promise._settlePromiseAt (/home/travis/.nvm/v0.10.36/lib/node_modules/eclint/node_modules/editorconfig/node_modules/bluebird/js/main/promise.js:577:18)
    at Async._drainQueue (/home/travis/.nvm/v0.10.36/lib/node_modules/eclint/node_modules/editorconfig/node_modules/bluebird/js/main/async.js:128:12)
    at Async._drainQueues (/home/travis/.nvm/v0.10.36/lib/node_modules/eclint/node_modules/editorconfig/node_modules/bluebird/js/main/async.js:133:10)
    at Async.drainQueues (/home/travis/.nvm/v0.10.36/lib/node_modules/eclint/node_modules/editorconfig/node_modules/bluebird/js/main/async.js:15:14)
    at process._tickCallback (node.js:442:13)

Copied from jednano/eclint#30.

Export doesn't return the specified API anymore

The export of the linez package does not return the API anymore, but an object with the API as the default property.

Should:

{ [Function: linez]
  Document: [Function: Document],
  configure: [Function: configure],
  resetConfiguration: [Function: resetConfiguration] }

Is:

{ default: 
   { [Function: linez]
     Document: [Function: Document],
     configure: [Function: configure],
     resetConfiguration: [Function: resetConfiguration] } }

This breaks for example the usage in the eclint package:

Unhandled rejection TypeError: linez is not a function
  at /xxx/node_modules/eclint/js/eclint.js:114:27

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.