kitura / kitura-couchdb Goto Github PK
View Code? Open in Web Editor NEWCouchDB adapter for Kitura
License: Apache License 2.0
CouchDB adapter for Kitura
License: Apache License 2.0
Previously Cloudant, IBM's branded CouchDB service switched from using username/password authentication to using IAM apikey authentication. For the time being, Cloudant provides both sets of credentials when users select Legacy and IAM credentials as opposed to IAM credentials.
End of life support from Cloudant for the legacy credentials has not yet been announced, but we should support service instances that are IAM-only. https://console.bluemix.net/docs/services/Cloudant/guides/iam.html#ibm-cloud-identity-and-access-management-iam-
I have an issue with the db.update method, it is never called if it is inside another db closure.
If i retrieve an object from the db with an id, update it and call db.update the update's callback method is never called.
Also, my mac freezed and when i took a look at the swapped memory, it was a few dozens gb, check the attached image.
I have created a gist for this: https://gist.github.com/lucianboboc/24a95fbc7234dbada8b80b1df23ffe2a
The problem I am facing is that I have a list of (ID, Revision) pairs, and I would like to delete all of these documents or update them in batch. This can be pulled off by iterating through each of these (ID, rev) pairs and calling the delete command each time. The problem with this approach is that each of those deletes are asynchronous, therefore it is immensely difficult to determine when all of them have completed. A promise framework or something similar could be useful here.
CouchDB has an API where you can request multiple documents are once:
http://docs.couchdb.org/en/2.2.0/api/database/bulk-api.html#db-bulk-get
We currently have support to Bulk insert or update documents but not to bulk retrieve them.
This API would be useful when you need to request lots of documents with known ID but don't want to make lots of API requests.
Travis-CI is reporting that my pull request is failing with the following error:
$ git clone -b master https://$GITHUB_USER:[email protected]/IBM-Swift/DevOps.git
Cloning into 'DevOps'...
remote: Repository not found.
fatal: Authentication failed for 'https://:@github.com/IBM-Swift/DevOps.git/'
The command "git clone -b master https://$GITHUB_USER:[email protected]/IBM-Swift/DevOps.git" failed and exited with 128 during .
Your build has been stopped.
This is much needed for the newest version of BluePic (blocked by this).
Attempting to use startKey or endKey as an AnyObject array on Linux (Swift binaries 4-25) results in compilation error: Contextual type 'AnyObject' cannot be used with array literal. The Swift compiler on Linux does not treat an AnyObject array as an AnyObject, and hence the compilation error (though on OS X, the compiler is OK with treating an AnyObject array as an AnyObject).
To address this issue, changing the startKey and endKey definitions to [AnyObject] and adding crucial logic for generating the {} characters.
Changes tested and now BluePic is no longer blocked by this.
The old travis.org service is no longer available.
You need to activate something in order to move the CI to travis-ci.com. PRs won't be able to get merged otherwise
#97 introduced the decodeDocuments
helper function, and adds comments that elude to the fact that when querying documents from CouchDB, you must set the include_docs
parameter to true
for Couch to include the documents themselves in the response. However, the existing queryByView
function does not document this requirement, and so it could be confusing that something like this does not work:
database.queryByView("fruitView", ofDesign: "fruitDesign", usingParameters: []) {
(documents, error) in
let fruitDocs = documents?.decodeDocuments(ofType: Fruit.self)
....
}
The solution in this case is to add usingParameters: [.includeDocs(true)]
. I feel this use-case should be discussed in the documentation for the queryByView function.
All source files contain the comment "Copyright © 2016 IBM. All rights reserved." which is at odds with the LICENSE.txt Apache 2.0 License.
When calling the method usersDatabase().createUser() on couchDBClient it will never return.
Is seems to get stuck in an endless loop in the network stack. The user will be created in couchDB, but the call never returns.
Here is my test code:
index f8bcf6b..c5f385d 100644
--- i/Sources/CouchDBSample/main.swift
+++ w/Sources/CouchDBSample/main.swift
@@ -179,8 +179,32 @@ func deleteDocument(revisionNumber: String) {
})
}
+func createUser() {
+ let newUser : [String:Any] = [
+ "name" : "testuser",
+ "type": "user",
+ "password": "apple",
+ "roles": []
+ ]
+
+ couchDBClient.usersDatabase().createUser(document: JSON(newUser)) {
+ id, doc, error in
+ guard let id = id,
+ let doc = doc,
+ error == nil else {
+ Log.error("User could not be created due to error: (error.description())")
+ return
+ }
+ Log.info("created user \(id) with doc: \(doc.debugDescription)")
+ }
+}
+
+
+
// Start tests...
-createDocument()
+//createDocument()
+
+createUser()
Log.info("Sample program completed its execution.")
I run it against a fresh, local couchDB 2.0 instance.
The stack trace is:
#1 0x00000001001f0eab in HTTPIncomingMessage.parsingCompleted() -> () at /Users/dominik/Developer/Colorimetrix/BunBun-Server/Packages/Kitura-net-1.4.3/Sources/KituraNet/HTTP/HTTPIncomingMessage.swift:253
#2 0x00000001001eec8d in HTTPIncomingMessage.parse(NSData, from : Int, completeBuffer : Bool) -> HTTPParserStatus at /Users/dominik/Developer/Colorimetrix/BunBun-Server/Packages/Kitura-net-1.4.3/Sources/KituraNet/HTTP/HTTPIncomingMessage.swift:169
#3 0x00000001001c38e7 in ClientResponse.parse() -> HTTPParserStatus at /Users/dominik/Developer/Colorimetrix/BunBun-Server/Packages/Kitura-net-1.4.3/Sources/KituraNet/ClientResponse.swift:57
#4 0x00000001001bd68d in ClientRequest.end(close : Bool) -> () at /Users/dominik/Developer/Colorimetrix/BunBun-Server/Packages/Kitura-net-1.4.3/Sources/KituraNet/ClientRequest.swift:344
#5 0x00000001001bceee in ClientRequest.end(String, close : Bool) -> () at /Users/dominik/Developer/Colorimetrix/BunBun-Server/Packages/Kitura-net-1.4.3/Sources/KituraNet/ClientRequest.swift:291
#6 0x00000001003560e1 in UsersDatabase.createUser(document : JSON, callback : (String?, JSON?, NSError?) -> ()) -> () at /Users/dominik/Developer/Colorimetrix/BunBun-Server/Packages/Kitura-CouchDB-1.4.0/Sources/CouchDB/UsersDatabase.swift:55
#7 0x0000000100007c3d in createUser() -> () at /Users/dominik/Developer/Colorimetrix/BunBun-Server/Packages/Kitura-CouchDB-1.4.0/Sources/CouchDBSample/main.swift:199
#8 0x00000001000035ae in main at /Users/dominik/Developer/Colorimetrix/BunBun-Server/Packages/Kitura-CouchDB-1.4.0/Sources/CouchDBSample/main.swift:208
I tested with the latest released Kitura stack
CloudFoundryEnv-1.9.2
HeliumLogger-1.4.0
Kitura-1.4.2
Kitura-CouchDB-1.4.0
Kitura-TemplateEngine-1.4.0
Kitura-net-1.4.3
LoggerAPI-1.4.0
SSLService-0.12.10
Socket-0.12.16
SwiftyJSON-15.0.4
It's probably a Kitura-net problem, but I'm just starting using the stack and not familiar with the low level stuff.
When I add .stale(.OK) as query parameter I get status code 400 back.
I am using Kitura NIO
There are two things I had to do to fix it:
1 Change to lowercase ok:
/// Indicates when to update.
public enum StaleOptions {
/// CouchDB will not refresh the view even if it is stale.
case ok
/// CouchDB will update the view after the stale result is returned.
case updateAfter
}
instead of case OK
2: Remove \" ... \"
from to string switch:
for param in params {
switch param {
case .conflicts (let value):
paramString += "conflicts=\(value)&"
…
case .stale (let value):
paramString += "stale=\(value)&"
case .startKey (let value):
…
}
}
instead of paramString += "stale=\"\(value)\"&"
After adding Kitura-CouchDB to dependencies project is unable to build
Package.swift:
let package = Package(
name: "KituraTest",
dependencies: [
.Package(url: "https://github.com/IBM-Swift/Kitura.git", majorVersion: 1, minor: 1),
.Package(url: "https://github.com/IBM-Swift/Kitura-CouchDB.git", majorVersion: 1, minor: 0),
]
)
Log:
$ swift build
Cloning https://github.com/IBM-Swift/Kitura.git
HEAD is now at 706108f Update to Swift 3.0.1 (#834)
Resolved version: 1.1.0
Cloning https://github.com/IBM-Swift/Kitura-net.git
HEAD is now at a3914e6 Update to Swift 3.0.1 (#138)
Resolved version: 1.1.0
Cloning https://github.com/IBM-Swift/LoggerAPI.git
HEAD is now at c64ce64 Merge pull request #16 from IBM-Swift/issue_832
Resolved version: 1.1.0
Cloning https://github.com/IBM-Swift/BlueSocket.git
HEAD is now at 61d47f7 Update to latest (11/1) toolchain.
Resolved version: 0.11.39
Cloning https://github.com/IBM-Swift/CCurl.git
HEAD is now at 3cfb752 Add header callback helper function (#9)
Resolved version: 0.2.3
Cloning https://github.com/IBM-Swift/CHTTPParser.git
HEAD is now at 429eff6 Merge pull request #7 from ianpartridge/master
Resolved version: 0.3.0
Cloning https://github.com/IBM-Swift/BlueSSLService.git
HEAD is now at 6659ac8 Update to latest (11/1) toolchain.
Resolved version: 0.11.53
Cloning https://github.com/IBM-Swift/SwiftyJSON.git
HEAD is now at ba99cbf Merge pull request #26 from IBM-Swift/issue_832
Resolved version: 15.0.0
Cloning https://github.com/IBM-Swift/Kitura-TemplateEngine.git
HEAD is now at 4fe7829 Support Swift 3.0.1 (#9)
Resolved version: 1.1.0
Compile CHTTPParser utils.c
Compile CHTTPParser http_parser.c
Compile Swift Module 'KituraTemplateEngine' (1 sources)
Compile Swift Module 'SwiftyJSON' (2 sources)
Compile Swift Module 'Socket' (3 sources)
Compile Swift Module 'LoggerAPI' (1 sources)
Linking CHTTPParser
Compile Swift Module 'SSLService' (1 sources)
Compile Swift Module 'KituraNet' (33 sources)
Compile Swift Module 'Kitura' (40 sources)
Compile Swift Module 'KituraTest' (1 sources)
Linking ./.build/debug/KituraTest
iMac5k-Sergei-2:KituraTest mak$ swift package generate-xcodeproj
Cloning https://github.com/IBM-Swift/Kitura-CouchDB.git
HEAD is now at ee48544 updated dependency versions in Package.swift
Resolved version: 1.0.0
Cloning https://github.com/IBM-Swift/HeliumLogger.git
warning: unable to rmdir Package-Builder: Directory not empty
HEAD is now at 4a52f0b updated dependency versions in Package.swift
Resolved version: 1.0.0
error: The dependency graph could not be satisfied (https://github.com/IBM-Swift/LoggerAPI.git)
iMac5k-Sergei-2:KituraTest mak$ rm -rf .build
iMac5k-Sergei-2:KituraTest mak$ swift build
Cloning https://github.com/IBM-Swift/Kitura.git
HEAD is now at 706108f Update to Swift 3.0.1 (#834)
Resolved version: 1.1.0
Cloning https://github.com/IBM-Swift/Kitura-net.git
HEAD is now at a3914e6 Update to Swift 3.0.1 (#138)
Resolved version: 1.1.0
Cloning https://github.com/IBM-Swift/LoggerAPI.git
HEAD is now at c64ce64 Merge pull request #16 from IBM-Swift/issue_832
Resolved version: 1.1.0
Cloning https://github.com/IBM-Swift/BlueSocket.git
HEAD is now at 61d47f7 Update to latest (11/1) toolchain.
Resolved version: 0.11.39
Cloning https://github.com/IBM-Swift/CCurl.git
HEAD is now at 3cfb752 Add header callback helper function (#9)
Resolved version: 0.2.3
Cloning https://github.com/IBM-Swift/CHTTPParser.git
HEAD is now at 429eff6 Merge pull request #7 from ianpartridge/master
Resolved version: 0.3.0
Cloning https://github.com/IBM-Swift/BlueSSLService.git
HEAD is now at 6659ac8 Update to latest (11/1) toolchain.
Resolved version: 0.11.53
Cloning https://github.com/IBM-Swift/SwiftyJSON.git
HEAD is now at ba99cbf Merge pull request #26 from IBM-Swift/issue_832
Resolved version: 15.0.0
Cloning https://github.com/IBM-Swift/Kitura-TemplateEngine.git
HEAD is now at 4fe7829 Support Swift 3.0.1 (#9)
Resolved version: 1.1.0
Cloning https://github.com/IBM-Swift/Kitura-CouchDB.git
HEAD is now at ee48544 updated dependency versions in Package.swift
Resolved version: 1.0.0
Cloning https://github.com/IBM-Swift/HeliumLogger.git
warning: unable to rmdir Package-Builder: Directory not empty
HEAD is now at 4a52f0b updated dependency versions in Package.swift
Resolved version: 1.0.0
error: The dependency graph could not be satisfied (https://github.com/IBM-Swift/LoggerAPI.git)
Kitura-CouchDB/Tests/CouchDB/Utils.swift:55: error: -[CouchDBTestSuite.DBTests testDB] : failed - Error in credentials.json.
Currently there is no chance to define a custom dateEncodingStrategy. The JSONEncoder object is created without any config options.
class func documentRequest<D: Document>(document: D, options: [ClientRequest.Options], callback: @escaping (DocumentResponse?, CouchDBError?) -> ()) {
do {
let requestBody = try JSONEncoder().encode(document)
couchRequest(body: requestBody, options: options, passStatusCodes: [.created, .accepted], callback: callback)
} catch {
return callback(nil, CouchDBError(HTTPStatusCode.internalServerError, reason: error.localizedDescription))
}
}
Would be great if we can define a custom date encoding strategy like this.
let encoder = JSONEncoder()
encoder.dateEncodingStrategy = .custom({ (date, encoder) in
var container = encoder.singleValueContainer()
try container.encode(DateFormatter.myDateFormatter.string(from: date))
})
What do you think about that?
Hey guys,
I have a problem with the queryByView method.
queryByView(_ view: String, ofDesign design: String, usingParameters params: [Database.QueryParameters], callback: @escaping (JSON?, NSError?) -> ())
It seems to ignore any key parameters using cloudant db. For example using key "[email protected]" results in getting all documents and not only the doc with key "[email protected]". It is working the right way using CouchDB 2.0.
Any Idea ?
Best regards
Marcel
When using both Kitura and CouchDB as dependencies the PM enters an infinite loop.
These are the dependencies I tried it with (clean package init with only these two):
dependencies: [
.package(url: "https://github.com/IBM-Swift/Kitura-CouchDB.git", .upToNextMinor(from: "2.0.0")),
.package(url: "https://github.com/IBM-Swift/Kitura.git", .upToNextMinor(from: "2.0.0"))
],
Looking at the logs (swift build -v) it seems to be related to BlueSocket somehow
UPDATE:
When trying with the previous rev it works fine (dcdc610):
.package(url: "https://github.com/IBM-Swift/Kitura-CouchDB.git", .revision("dcdc61061553306226c92768c3b44485379e39a7")),
.package(url: "https://github.com/IBM-Swift/Kitura.git", .upToNextMinor(from: "2.0.0"))
I am wondering if there is a way to use this repository and avoid the Callback Hell?
Promises or Swift5 Result
would be good candidates to do so IMO. Are there any implementation plans?
The generated API docs contain the LoggerAPI
docs instead of Kitura-CouchDB
docs. Need to regenerate docs with the correct jazzy
settings/arguments.
Currently the CouchDBClient is created with a Username and password which is attached to every request using basic authentication.
It is more performant and safe to use cookies as described in the couchDB docs:
http://docs.couchdb.org/en/stable/api/server/authn.html#cookie-authentication
We have a function for creating users and sessions however their is no ability to use the cookie from the session as authentication for requests.
If we added this feature then you could add authentication levels to users instead of everyone having the authorization level of the provided username and password.
There was a missing return statement in the CouchDB tests. This was causing the tests to automatically pass without actually running.
A pr fixing this has been raised here
Now the tests are running the KituraNio travis builds are failing.
We need to investigate why they are failing and either fix them or propose a new solution for making rest requests within this repo.
I have the following view :
let design = DesignDocument(_id: "_design/mydesign", views: [
"count_doc": [
"map": "function(doc) { if(doc.type == 'mydoc') { emit(doc._id, 1); } }",
"reduce": "_count"
]
])
in version 2.1.0 I was able to get the result with the following code:
database.queryByView("count_doc", ofDesign: "mydesign", usingParameters: []) { (doc, err) in
if let doc = doc, err == nil {
if let count = doc["rows"][0]["value"].int {
completion(count, nil)
} else {
completion(0, nil)
}
} else {
completion(nil, err)
}
}
Now in the current version:
database?.queryByView("count_doc", ofDesign: "mydesign", usingParameters: []) { (doc, err) in
if let doc = doc, err == nil {
if let count = doc.total_rows {
completion(count, nil)
} else {
completion(0, nil)
}
} else {
completion(nil, err)
}
}
I get the error
Failed to decode AllDatabaseDocuments from response
which is correct since the result is something like
{"rows":[ {"key":null,"value":26} ]}
Can you tell me how to retrieve the result from this view or a view that contains "_stats" function ?
So, I know databases like sql or MongoDB, but here, I don't how to filter values or do a simple db query? Can you update this, in the documentation ?
When attempting to run CouchDBSample like so:
.build/debug/CouchDBSample localhost kiturauser 6SZ-X4Q-Hyv-pBb
I get the following error:
Starting sample program...
connPropertiesStr:
hostName -> localhost
port -> 80
secured -> false
userName -> Optional("kiturauser")
password -> Optional("6SZ-X4Q-Hyv-pBb")
Hostname is: localhost
ClientRequest Error. CURL Return code=CURLcode(rawValue: 7)
>> Oops something went wrong; could not persist document.
0
CouchDBDomain
[NSLocalizedDescription: Internal Error]
Sample program completed its execution.
I've manually changed the database name that it tries to connect to:
// Create database instance to perform any document operations
let database = couchDBClient.database("kitura_test_db")
I also have a CouchDB database set up with the same name.
I get this error with both master
and develop
branches.
Perhaps this identifies some strange differences between Cloudant and CouchDB, but we were using a field that is a boolean. All of our tests pass when using CouchDB, but they failed when using Cloudant. We changed the value from using a native boolean type to an integer (0 or 1 for false and true respectively) and the issue was resolved.
I have a +
character in some of my _id
's
I am able to create a document with +
as part of the _id
. Then if I try to retrieve by the same id, it is not found.
let id = "abc+1"
let document = Document(id: id)
database.create(document) { response, error in
// document created with ok (can see it using the Fauxton UI)
// Try to get it.
database.retrieve(id) { document, error in
// error.statusCode will be .notFound here
}
}
I noticed in HTTP.swift
/// A set of characters that are valid in requests.
private static let allowedCharacterSet = NSCharacterSet(charactersIn:"\"#%/<>?@\\^`{|} ").inverted
Is the right thing to do add it to here? of it there a reason we don't have +
in the allowed character set?
I'm trying to retrieve all documents in the database. While I can easily use "_all_docs" as the id argument, it only returns the documents without the docs' values, which in my case I need. I know the "?" gets escaped out as part of Kitura Net, but I'm wondering if there is a way around this perhaps. The error I get is: Error: not_found, reason: missing Code: 404. I know if I don't escape the ?, it works.
Hello i have a simple queryByView up in the cloundant/ibm platform with 3 users in the db. The more users the more it returns nil.
the error that returns: "the operation couldn't be completed"
database.queryByView("users-view", ofDesign: "users",
usingParameters: [.includeDocs(true)],
the problem is sometimes it returns nil and sometimes does return the 3 users, this problem doesn't happen in localhost.
map function:
function(doc) {
if (doc.type == 'user')
{
emit(doc);
}}
The port
parameter for ConnectionProperties
is an Int16
which seems a bit specific. I'm currently having to convert a variable of type Int
to Int16
which seems like "busy work"...
let connectionProperties = ConnectionProperties(
host: host,
port: Int16(port),
secured: secured
)
Can we make it an Int
?
Hi I want to make a big database for eg in SQL I have tables for login, customer, projects, etc
how to achieve this on CouchDB
When preparing the request, the schema (aka protocol) value should be obtained from ConnectionProperties: HTTPProtocol. This way Kitura-CouchDB can connect using the right protocol, http or https.
I get this error when running 'swift build' in terminal of Macbook. What I suppose to change to fix it?
'KituraTIL' /Users/Chuck/KituraTIL: error: dependency 'CouchDB' in target 'KituraTIL' requires explicit declaration; reference the package in the target dependency with '.product(name: "CouchDB", package: "Kitura-CouchDB")'
`
// swift-tools-version:5.2
// The swift-tools-version declares the minimum version of Swift required to build this package.
import PackageDescription
let package = Package(
// 1
name: "KituraTIL",
dependencies: [
// 2
.package(url: "https://github.com/IBM-Swift/Kitura.git",
.upToNextMajor(from: "2.0.0")),
// 3
.package(url: "https://github.com/IBM-Swift/HeliumLogger.git",
.upToNextMajor(from: "1.0.0")),
// 4
.package(url: "https://github.com/IBM-Swift/Kitura-CouchDB.git",
.upToNextMajor(from: "3.0.0"))
],
//5
targets: [
.target(name: "KituraTIL",
dependencies: ["Kitura" , "HeliumLogger", "CouchDB"],
path: "Sources")
]
)
`
Are there any plans to move from using SwiftyJSON to supporting the Codable protocol for JSON within this proejct?
error: value of type 'String' does not conform to expected element type 'AnyObject'
case Keys ([Any])
#else
case Keys ([AnyObject])
#endif
case EndKey (Any)
#else
case EndKey (AnyObject)
#endif
case StartKey (Any)
#else
case StartKey (AnyObject)
#endif
public func queryByView(view: String, ofDesign design: String, usingParameters params: [Database.QueryParameters], callback: (JSON?, NSError?) -> ()) {
var paramString = ""
#if os(Linux)
var keys: [Any]?
#else
var keys: [AnyObject]?
#endif
// other code goes here.
}
Several places in source and samples makes references to "pheonix". These should be renamed to "kitura".
While using Kitura-CouchDB
and executing REST requests through its API, we are running into the following issue:
[ClientRequest.swift:360 end(close:)] ClientRequest error. Failed to parse response. Error=Unexpectedly got an EOF when reading the request.
I have created a very small program that can be used to easily and consistently reproduce this problem without effort. I am about to create a repo with this sample/test program and share here the steps for reproducing the issue.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.