fent / feedme.js Goto Github PK
View Code? Open in Web Editor NEWRSS/Atom/JSON feed parser
License: MIT License
RSS/Atom/JSON feed parser
License: MIT License
The following error is occurring when trying to parse a JSON Feed.
Failing tested JSON Feeds:
https://daringfireball.net/feeds/json
http://inessential.com/feed.json
Error:
Exception has occurred: Error
TypeError: Cannot set property 'type' of undefined
at CStream.parser.done (e:\Projects\ely\node_modules\feedme\lib\jsonfeedparser.js:140:15)
at self.children.forEach (e:\Projects\ely\node_modules\eventyoshi\lib\index.js:255:28)
at Map.forEach (native)
at EventYoshi.Array.slice.call.forEach.self.(anonymous function) [as done] (e:\Projects\ely\node_modules\eventyoshi\lib\index.js:253:23)
at EventYoshi.<anonymous> (e:\Projects\ely\feedparser.js:15:27)
at emitNone (events.js:86:13)
at EventYoshi.emit (events.js:185:7)
at CStream.wrapper.__wrappers.(anonymous function) (e:\Projects\ely\node_modules\eventyoshi\lib\index.js:105:14)
at emitNone (events.js:86:13)
at CStream.emit (events.js:185:7)
This is the e:\Projects\ely\feedparser.js
:
var FeedMe = require('feedme');
var Request = require('request');
var URL = require('url');
module.exports = function (url, cb) {
var feedme = new FeedMe(true);
feedme.on('error', function (error) {
return cb(415);
});
feedme.on('end', function () {
var feed = feedme.done();
return cb(null, feed);
});
var options = {
encoding: null,
gzip: true,
timeout: 5000,
url: URL.parse(url)
};
Request.get(options, function(error, response, body) {
if (error) {
return cb(response.statusCode);
} else {
feedme.write(body);
feedme.end();
}
})
};
Hello!
It seems that when using buffering, and piping an HTTP stream, the 'end' event is emitted twice. I wrote a small gist to highlight the issue.
HTH.
As a notable dependent on clarinet you may wish to offer input on a breaking API change I am proposing for the next major version of clarinet (dscape/clarinet#47).
Should the proposal be acted upon, based on your current package.json
no action should be needed until explicitly upgrading dependencies past current version targets.
Branch | Build failing 🚨 |
---|---|
Dependency | mocha |
Current Version | 3.5.3 |
Type | devDependency |
This version is covered by your current version range and after updating it in your project the build failed.
As mocha is “only” a devDependency of this project it might not break production or downstream projects, but “only” your build or test tools – preventing new deploys or publishes.
I recommend you give this issue a high priority. I’m sure you can resolve this 💪
You might want to read this before filing a new bug! 😝
For more info, please read this article.
node
from exiting will do so when run in Mocha. Supply the --exit
flag to revert to pre-v4.0.0 behavior (@ScottFreeCode, @boneskull)stdout:
prefix from browser reporter logs (@skeggse)--forbid-pending
or --forbid-only
is specified (@ScottFreeCode)--compilers
command-line option is now soft-deprecated and will emit a warning on STDERR
. Read this for more info and workarounds (@ScottFreeCode, @boneskull)The new version differs by 48 commits.
d69bf14
Release v4.0.0
171b9f9
pfix "prepublishOnly" potential portability problem
60e39d9
Update link to wiki (GitHub at the leading --
)
804f9d5
Update link because GitHub ate the leading --
3326c23
update CHANGELOG for v4.0.0 [ci skip]
6dd9252
add link to wiki on --compilers deprecation
96318e1
Deprecate --compilers
92beda9
drop bower support
58a4c6a
remove unused .npmignore
7af6611
kill Date#toISOString shim
43501a2
reduce noise about slow tests; make a few tests faster, etc.
fa228e9
update --exit / --no-exit integration test for new default behavior
3fdd3ff
Switch default from forced exit to no-exit
c5d69e0
add integration tests for --exit/--no-exit
3a7f8dc
enhance runMochaJSON() helper by returning the subprocess instance
There are 48 commits in total.
See the full diff
There is a collection of frequently asked questions and of course you may always ask my humans.
Your Greenkeeper Bot 🌴
First of: awesome work! Really saved me a lot of trouble.
I'm getting warnings from EventYoshi:
Trace
at addListener (events.js:239:17)
at EventYoshi.on (/Users/Michael/Code/MagicMirror/node_modules/eventyoshi/lib/index.js:146:15)
at EventYoshi.once (/Users/Michael/Code/MagicMirror/node_modules/eventyoshi/lib/index.js:170:8)
at IconvLiteDecoderStream.Readable.pipe (_stream_readable.js:568:8)
at self.fetchNews (/Users/Michael/Code/MagicMirror/modules/default/newsfeed/newsfetcher.js:49:76)
at fetchNews (/Users/Michael/Code/MagicMirror/modules/default/newsfeed/fetcher.js:40:15)
at .<anonymous> (/Users/Michael/Code/MagicMirror/modules/default/newsfeed/fetcher.js:58:4)
at Timer.listOnTimeout (timers.js:92:15)
(node) warning: possible EventEmitter memory leak detected. 11 finish listeners added. Use emitter.setMaxListeners() to increase limit.
The mentioned number of listeners (11) does not increase. So it's doesn't seem like a real issue. Maybe it's a good idea to increase the maxListeners?
getting this error
Error: status code 301
at ClientRequest. (F:\rohit work\repopushnotifications\pushnotificationsqualzzbackend\src\rss_feeds\rss_feeds.controller.ts:38:23)
/////////////////////////////////////////////////////
code I am using nest.js node js frame work
@Get('/rss') async GetFeeds() { console.log("abc"); http.get('http://www.npr.org/rss/rss.php?id=1001', (res:any) => { if (res.statusCode != 200) { console.error(new Error(
status code ${res.statusCode})); return; } let parser = new FeedMe(); parser.on('title', (title:any) => { console.log('title of feed is', title); }); parser.on('item', (item:any) => { console.log(); console.log('news:', item.title); console.log(item.description); }); res.pipe(parser); },(err:any)=>{ console.log(err); }); }
I'm trying to parse an Atom feed, but somehow, FeedMe never fires an event on 'entry'.
To test it, I tried the following code:
var FeedMe = require("feedme");
var request = require("request");
var parser = new FeedMe();
// Print the feed type. This is called just fine.
parser.on("type", function(type) {
console.log('Feed type: ' + type);
});
// This is never called, while the feed contains entry items
parser.on("entry", function(entry) {
console.log(entry);
});
// This is called just fine
parser.on("end", function() {
console.log('End of feed reached.');
});
parser.on("error", function(error) {
console.log('Error parsing feed: ');
console.log(error);
});
request({uri: 'http://www.reddit.com/r/technology/.rss'}).pipe(parser);
console.log("Request started!");
Am I doing something wrong?
To see what happens to your code in Node.js 10, Greenkeeper has created a branch with the following changes:
.travis.yml
package.json
files, so that was left aloneIf you’re interested in upgrading this repo to Node.js 10, you can open a PR with these changes. Please note that this issue is just intended as a friendly reminder and the PR as a possible starting point for getting your code running on Node.js 10.
Greenkeeper has checked the engines
key in any package.json
file, the .nvmrc
file, and the .travis.yml
file, if present.
engines
was only updated if it defined a single version, not a range..nvmrc
was updated to Node.js 10.travis.yml
was only changed if there was a root-level node_js
that didn’t already include Node.js 10, such as node
or lts/*
. In this case, the new version was appended to the list. We didn’t touch job or matrix configurations because these tend to be quite specific and complex, and it’s difficult to infer what the intentions were.For many simpler .travis.yml
configurations, this PR should suffice as-is, but depending on what you’re doing it may require additional work or may not be applicable at all. We’re also aware that you may have good reasons to not update to Node.js 10, which is why this was sent as an issue and not a pull request. Feel free to delete it without comment, I’m a humble robot and won’t feel rejected 🤖
There is a collection of frequently asked questions. If those don’t help, you can always ask the humans behind Greenkeeper.
Your Greenkeeper Bot 🌴
I've tried to use the library for fetching a remote url, but failed setting up the stream correctly. An example with such a case might be useful for future users. Here is one the things I tried:
import FeedMe from 'feedme';
import request from 'request';
const extractedData = [];
const feedUrl = 'http://example.com/feed.rss';
const feedParser = new FeedMe(true);
feedParser.on('item', (feedItem) => {
// push a new object to extractedData
console.log('item');
});
feedParser.on('end', () => {
// save extractedData
console.log('end');
});
request(feedUrl).pipe(feedParser);
For some reason, the output is always the following:
end
item
item
...
Could you please provide the right way of fetching the feed remotely? I imagine this case is used more often than the one with a local fs
.
https://feeds.pinboard.in/rss/t:hardware
AFAICT its a valid feed (https://validator.w3.org/feed/check.cgi?url=https%3A%2F%2Ffeeds.pinboard.in%2Frss%2Ft%3Ahardware),
and the underlying site is in fact giving me a usable status (200).
Here an example https://www.smashingmagazine.com/feed/
While e.g. this parsing algorithm https://github.com/Xstoudi/alduin/blob/master/src/app/js/util/feed-parser.ts processes it fine
I'd like to return the feed (e.g.http://feeds.bbci.co.uk/news/world/asia/rss.xml) as Javascript object. Do you have an example of using parser.done()
? Thanks!
Eg. http://backend.deviantart.com/rss.xml?q=favby:bootz15&type=deviation&offset=0
Looks like legit RSS to me.
Example code crashes on way to many RSS links. For example.
Can you add an example that outputs a JSON output to the command line or anywhere? Doesn't seem to work when I do it.
I've got the following error parsing http://www.blogger.com/feeds/1350160888577517049/posts/default?start-index=151&max-results=150
events.js:167
throw er; // Unhandled 'error' event
^
Error: Feed type not recognized
If there is a tag like this, inside a rss item, it is parsed as an empty object not as an empty string
I'm getting this error that causes nodejs to crash when feed type is nog recognized. The feed that causes this issue is:
https://www.dadapixel.blog/?format=rss
Output:
Error: Feed type not recognized
at SAXStream.openf1 (C:\Data\Working\REJH_SVN\feedlr\src\nodejs\fdlr-reader\node_modules\feedme\lib\xmlfeedparser.js:106:28)
at emitOne (events.js:96:13)
at SAXStream.emit (events.js:188:7)
at Object.me._parser.(anonymous function) [as onopentag] (C:\Data\Working\REJH_SVN\feedlr\src\nodejs\fdlr-reader\node_modules\sax\lib\sax.js:258:17)
at emit (C:\Data\Working\REJH_SVN\feedlr\src\nodejs\fdlr-reader\node_modules\sax\lib\sax.js:640:35)
at emitNode (C:\Data\Working\REJH_SVN\feedlr\src\nodejs\fdlr-reader\node_modules\sax\lib\sax.js:645:5)
at openTag (C:\Data\Working\REJH_SVN\feedlr\src\nodejs\fdlr-reader\node_modules\sax\lib\sax.js:841:5)
at Object.write (C:\Data\Working\REJH_SVN\feedlr\src\nodejs\fdlr-reader\node_modules\sax\lib\sax.js:1294:15)
at SAXStream.write (C:\Data\Working\REJH_SVN\feedlr\src\nodejs\fdlr-reader\node_modules\sax\lib\sax.js:239:18)
at yoshi.write (C:\Data\Working\REJH_SVN\feedlr\src\nodejs\fdlr-reader\node_modules\feedme\lib\feedme.js:41:14)
C:\Data\Working\REJH_SVN\feedlr\src\nodejs\fdlr-reader\node_modules\feedme\lib\xmlfeedparser.js:107
parser.close();
^TypeError: parser.close is not a function
at SAXStream.openf1 (C:\Data\Working\REJH_SVN\feedlr\src\nodejs\fdlr-reader\node_modules\feedme\lib\xmlfeedparser.js:107:14)
at emitOne (events.js:96:13)
at SAXStream.emit (events.js:188:7)
at Object.me._parser.(anonymous function) [as onopentag] (C:\Data\Working\REJH_SVN\feedlr\src\nodejs\fdlr-reader\node_modules\sax\lib\sax.js:258:17)
at emit (C:\Data\Working\REJH_SVN\feedlr\src\nodejs\fdlr-reader\node_modules\sax\lib\sax.js:640:35)
at emitNode (C:\Data\Working\REJH_SVN\feedlr\src\nodejs\fdlr-reader\node_modules\sax\lib\sax.js:645:5)
at openTag (C:\Data\Working\REJH_SVN\feedlr\src\nodejs\fdlr-reader\node_modules\sax\lib\sax.js:841:5)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.