Comments (7)
Hi there. Could you please post the code you are using, including all options passed to the search()
function (such as locationId
)?
Also, are there results when you perform the same search on the Kijiji website?
from kijiji-scraper.
If you sub in the category ID to 79 instead of 27 in the example code, you get the same error.
Applies with any other service ID.
No issues seeing the actual results on the Kijiji webiste.
E.g.
const kijiji = require("kijiji-scraper");
let options = {
minResults: 40
};
let params = {
locationId: 1700185, // Same as kijiji.locations.ONTARIO.OTTAWA_GATINEAU_AREA.OTTAWA
categoryId: 79, // Same as kijiji.categories.CARS_AND_VEHICLES
sortByName: "priceAsc" // Show the cheapest listings first
};
// Scrape using returned promise
kijiji.search(params, options).then(function(ads) {
// Use the ads array
for (let i = 0; i < ads.length; ++i) {
console.log(ads[i].title);
}
}).catch(console.error);
// Scrape using optional callback parameter
function callback(err, ads) {
if (!err) {
// Use the ads array
for (let i = 0; i < ads.length; ++i) {
console.log(ads[i].title);
}
}
}
kijiji.search(params, options, callback);
from kijiji-scraper.
Alright, I had a chance to look at this closer and it turns out that Kijiji uses slightly different markup on the results page for services compared to other categories. I'm not sure why or if they do this for other categories as well. It's possible they're in the process of transitioning to new markup site-wide. That's happened before and they changed it gradually for some ads, rather than site-wide.
Good find! I have a hacked together fix working locally and will be able to clean it up and test a bit more thoroughly tomorrow.
from kijiji-scraper.
That's pretty interesting - definitely wouldn't have figured that out myself. Thank you for putting all this together. Appreciate it!
from kijiji-scraper.
Hi, please check out the branch services-search-fix and try your search again using that. I did some testing and your issue should be resolved. I will merge, then publish to npm after confirmation.
from kijiji-scraper.
Sorry for the late reply. Thank you for the fix. It works perfectly. Tried a few different categories and haven't run into any issues. Awesome work!
from kijiji-scraper.
Great to hear! I've merged and the update is live on npm.
from kijiji-scraper.
Related Issues (20)
- Random Error:Invalid Kijiji HTML on search results page HOT 5
- Scraper fails to return search results (Kijiji scraper detection + blocking) HOT 23
- Search() with URL HOT 1
- "npm install" fails HOT 9
- MinResults and MaxResults in Search() function implemented improperly, Doesn't allow for getting every listing available. HOT 9
- kijiji.search() Randomly Returns an Amount of Ads Under Specified Amount via minResults HOT 4
- Consideration for expanding repo to Kijiji API HOT 1
- Kijiji.Ad.Get() Doesn't return to callback HOT 1
- Impossible to fetch ads HOT 5
- Scraper fails to properly scrape some numeric attributes (real numbers) HOT 6
- Missing very new ads when using the Mobile API HOT 15
- Ad #... does not exist HOT 2
- Incorrect Size Attribute For Shoe Category HOT 6
- Category Attributes HOT 5
- Create helper types for enums, and expose expected response attributes
- Package broken: Kijiji-scraper returns no results HOT 5
- Not working HOT 5
- Kijiji Mingle API Issue HOT 11
- How long is a temporary block? I haven't been able to get results since yesterday. HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from kijiji-scraper.