datanoise / mongo.cr Goto Github PK
View Code? Open in Web Editor NEWCrystal binding for MongoDB C Driver
License: MIT License
Crystal binding for MongoDB C Driver
License: MIT License
Hey, I was just wondering if anyone could possibly write an examples directory for this, so that we can actually see how to use this properly. There's seems to be a lot of little details getting in the way of using this effectively without a lot of mongoc lib knowledge. Also having to dig around in the codebase and guess what to do with a lot of the stuff is a bit of a nightmare.
Hello,
I want to list all entries of my collection.
Is there any way to do this ?
Regards,
hi, can i use upsert method in mongocr
Hi,
I'm trying to build this package on fedora
, since I have trouble using it:
First I install
dnf install libevent-devel pcre-devel gc-devel libmongo-client-devel
While downloading the code via git, I can not build this library crystal build src/mongo.cr
, running it lead me to a
/usr/lib/gcc/x86_64-redhat-linux/5.3.1/../../../../lib64/crt1.o: In function `_start':
(.text+0x20): undefined reference to `main'
cc compilation crashes
hi, i hope somebody can upgrade mongo-c driver that higher than 1.1.0
, i always fail when trying to make
mongo-c driver version 1.1.0
src/mongoc/mongoc-stream-tls.c:134:5: error: dereferencing pointer to incomplete type "Bio {aka struct bio_st}
By running the homepage example in a containerized environment I'm getting the following error:
Unhandled exception: File not open for writing (IO::Error)
from usr/share/crystal/src/crystal/system/unix/file_descriptor.cr:23:11 in 'unbuffered_write'
from usr/share/crystal/src/io/buffered.cr:205:5 in 'flush'
from usr/share/crystal/src/logger.cr:175:7 in 'write'
from usr/share/crystal/src/logger.cr:155:5 in 'log'
from src/lib/mongo/src/mongo.cr:35:18 in 'log'
from src/lib/mongo/src/mongo.cr:39:5 in '->'
from mongoc_log
from ???
from ???
from ???
from ???
from ???
from ???
from ???
from mongoc_cursor_next
from src/lib/mongo/src/mongo/cursor.cr:47:8 in 'next'
The containerized environment that I'm using is ubuntu:18.04 image with the following packages installed: libmongoc-dev
, libbson-dev
.
Is there some kind of setup that I'm missing?
I think it would be best if it lived in another repo and then added to the deps
block in Projectfile for this project.
In the comments for the collection insert
method, it says: "you can retrieve the generated _id from the last_error
method", however when I call the last_error
method I don't see where to get the _id.
https://github.com/datanoise/mongo.cr/blob/master/src/mongo/collection.cr#L149
Doc: https://docs.mongodb.com/manual/changeStreams/
Change streams allow applications to access real-time data changes without the complexity and risk of tailing the oplog.
Applications can use change streams to subscribe to all data changes on a collection and immediately react to them.
Note: I think this needs to resolve #29 first, to be able to do other things while waiting for a Change Event.
I'm trying to use the command()
method to query the connection status of the database, but the compiler seems to keep complaining about the fact the query should be of type BSON, but I'm unsure how to actually create a BSON object, I tried making use of the from_json
method on the BSON class, but couldn't get that to work. This is what I have so far:
def live?
-»»»begin
-»»»-»»»if @client.command("aggregation", {connectionStatus: 1})
-»»»-»»»-»»»return true
-»»»-»»»else
-»»»-»»»-»»»return false
-»»»-»»»end
-»»»rescue
-»»»-»»»return false
-»»»end
end
When I try to compile this I get:
Error in mxmongo.cr:14: instantiating 'MxMongo#live?()'
if mongo_client.live?
^~~~~
in classes/mx_mongo.cr:25: instantiating '(Mongo::Client | Nil:Class)#command(String, NamedTuple(connectionStatus: Int32))'
if @client.command("aggregation", {"connectionStatus": 1})
^~~~~~~
in lib/mongo/src/mongo/client.cr:40: instantiating 'command(String, NamedTuple(connectionStatus: Int32), BSON, LibMongoC::QueryFlags, Int32, Int32, Int32, Nil)'
def command(db_name, query, fields = BSON.new, flags = LibMongoC::QueryFlags::NONE,
^
in lib/mongo/src/mongo/client.cr:43: argument 'query' of 'LibMongoC#client_command' must be Pointer(LibBSON::BSONHandle), not NamedTuple(connectionStatus: Int32)
query, fields, prefs)
^~~~~
Maybe someone could help me with this, I installed everything but it is not working anyway.
Error:
/home/daniel/.cache/crystal/crystal-run-database.tmp: error while loading shared libraries: libmongoc-1.0.so.0: cannot open shared object file: No such file or directory
Environment: a Debian stretch up to date, with libmongoc-dev
package
> crystal --version
Crystal 0.22.0 [3c71228] (2017-04-20) LLVM 3.5.0
> llvm-config-3.9 --version
3.9.1
> cat /etc/{issue,debian_version}
Debian GNU/Linux 9 \n \l
9.0
> dpkg -l libmongoc-dev
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-===============================================-============================-============================-===================================================================================================
ii libmongoc-dev 1.4.2-1+b1 amd64 MongoDB C client library - dev files
From mongo.cr master branch.
> git clone https://github.com/datanoise/mongo.cr
Cloning into 'mongo.cr'...
remote: Counting objects: 355, done.
remote: Total 355 (delta 0), reused 0 (delta 0), pack-reused 354
Receiving objects: 100% (355/355), 78.30 KiB | 0 bytes/s, done.
Resolving deltas: 100% (203/203), done.
Checking connectivity... done.
> crystal spec
FF......................................................................
Failures:
1) Mongo::Uri should be able to create new uri
Failure/Error: host.port.should eq(27017)
Expected: 27017
got: 0
# spec/uri_spec.cr:10
2) Mongo::Uri should be able to create new uri with host and port
Failure/Error: host.port.should eq(27017)
Expected: 27017
got: 0
# spec/uri_spec.cr:18
Finished in 590.46 milliseconds
72 examples, 2 failures, 0 errors, 0 pending
Failed examples:
crystal spec spec/uri_spec.cr:5 # Mongo::Uri should be able to create new uri
crystal spec spec/uri_spec.cr:13 # Mongo::Uri should be able to create new uri with host and port
What I want: Find a Document by the Document ID.
My Code and tries:
Collection.update({ "_id" => { "$oid" => ANYVALIDID } }, { "$set" => { "title" => "Hello World" } })
# Not working
id = BSON::ObjectId.new(doc["id"].to_s)
Coolection.update({ "_id" => id }, { "title" => "Hello World" })
# Not working. Error:
# Index out of bounds (IndexError)
# 0x5d9b1f: *BSON#[]<String>:(BSON | BSON::Code | BSON::MaxKey | BSON::MinKey | BSON::ObjectId | # BSON::Symbol | BSON::Timestamp | Bool | Float64 | Int32 | Int64 | Regex | String | Time | Nil) at /app/lib/mongo/src/bson.cr 127:18
Any Ideas
@datanoise
Does mongo.cr use the asynchronous event loop provided by Crystal, or will it block?
@datanoise I need to construct a field within a document that contains an array of embedded documents. How can I do this? It looks like the provided BSON library does not support BsonArray?
Environment: a Debian stretch up to date, with libmongoc-dev
package
> crystal --version
Crystal 0.22.0 [3c71228] (2017-04-20) LLVM 3.5.0
> llvm-config-3.9 --version
3.9.1
> cat /etc/{issue,debian_version}
Debian GNU/Linux 9 \n \l
9.0
> dpkg -l libmongoc-dev
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name Version Architecture Description
+++-===============================================-============================-============================-===================================================================================================
ii libmongoc-dev 1.4.2-1+b1 amd64 MongoDB C client library - dev files
Code: an attempt to connect to a local Mongo database.
> crystal init app try_mongo
create try_mongo/.gitignore
create try_mongo/LICENSE
create try_mongo/README.md
create try_mongo/.travis.yml
create try_mongo/shard.yml
create try_mongo/src/try_mongo.cr
create try_mongo/src/try_mongo/version.cr
create try_mongo/spec/spec_helper.cr
create try_mongo/spec/try_mongo_spec.cr
Initialized empty Git repository in /path/to/try_mongo/.git/
Adding mongo.cr
> cd try_mongo
try_mongo> cat >> shard.yml
dependencies:
mongo:
github: datanoise/mongo.cr
branch: master
^D
try_mongo> shards install
Updating https://github.com/datanoise/mongo.cr.git
Installing mongo (master)
Add basic code to try db connect
try_mongo> cat > src/try_mongo.cr
require "mongo"
client = Mongo::Client.new "mongodb://localhost"
db = client["test"]
collection = db["mongo_try"]
puts collection.count
^D
Attempt to build...
> crystal deps build
Dependencies are satisfied
Building: try_mongo
Error target try_mongo failed to compile:
/usr/bin/ld: /opt/crystal/embedded/lib/../lib/libgc.a(os_dep.o): undefined reference to symbol '_end'
//usr/lib/x86_64-linux-gnu/libyajl.so.2: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
Error: execution of command failed with code: 1: `cc -o "try_mongo/bin/try_mongo" "${@}" -rdynamic -lmongoc-1.0 -lbson-1.0 -lpcre -lgc -lpthread /opt/crystal/src/ext/libcrystal.a -levent -lrt -ldl -L/usr/lib -L/usr/local/lib`
When I try to insert a doc with an array of tuple :
og = [{"property"=>"og:title","content"=>"Open Graph protocol"},{"property"=>"og:type","content"=>"website"}]
bookmarkCollection.insert({
"link" => "https://angular.io",
"og" => og,
"tags" => [firstTag, secondTag]
})
I've following error :
no overload matches 'BSON#[]=' with types String, (BSON | Mongo::Cursor | Nil)
- BSON#[]=(key, value : Int64)
- BSON#[]=(key, value : Binary)
- BSON#[]=(key, value : Bool)
- BSON#[]=(key, value : Float64 | Float32)
- BSON#[]=(key, value : MinKey)
- BSON#[]=(key, value : MaxKey)
- BSON#[]=(key, value : Nil)
- BSON#[]=(key, value : ObjectId)
- BSON#[]=(key, value : String)
- BSON#[]=(key, value : Symbol)
- BSON#[]=(key, value : Time)
- BSON#[]=(key, value : Timestamp)
- BSON#[]=(key, value : Code)
- BSON#[]=(key, value : BSON)
- BSON#[]=(key, value : Regex)
Couldn't find overloads for these types:
- BSON#[]=(key : String, value : Mongo::Cursor)
bson[i.to_s] = item
So I inserted following data into the database:
{
"name": "Group #1",
"users": [
"daniel",
"bob"
]
}
But if I want to get that array back, the Result is an BSON::Code
and I can't do anything with it. How could that be fixed and is there any workaround? @datanoise
When trying to execute the following query
{ $and: [ { "created_at": { $gte: ISODate("2020-05-01T00:00:00.000+0000") } }, { "created_at": { $lt: ISODate("2020-06-01T00:00:00.000+0000") } } ] }
I've tried
collection.find{ $and: [ { "created_at": { $gte: ISODate("2020-05-01T00:00:00.000+0000") } }, { "created_at": { $lt: ISODate("2020-06-01T00:00:00.000+0000") } } ] }.each do |doc|
docs << doc
p doc
end
This generates the following error
Error: $global_variables are not supported, use @@class_variables instead
I've tried quoting the globals, e.g..
collection.find{ "$and": [ { "created_at": { "$gte": ISODate("2020-05-01T00:00:00.000+0000") } }, { "created_at": { "$lt": ISODate("2020-06-01T00:00:00.000+0000") } } ] }.each do |doc|
docs << doc
p doc
end
This generates
Error: unexpected token: :
Any suggestions?
Many thanks!
Invalid memory access (signal 11) at address 0xa0
[4496293] *CallStack::print_backtrace:Int32 +117
[4468520] __crystal_sigfault_handler +56
[4875304] sigfault_handler +40
[139785657689056] ???
[139785662203494] ???
[139785662203690] mongoc_client_new +58
[4756622] *Mongo::Client#initialize:Nil +30
[4756561] *Mongo::Client::new:Mongo::Client +81
[4450987] ???
Getting a problem when trying to create an instance of a collection, I can't get past the compilation as I think the compiler is trying to account for the client instance possibly being 'Nil' and there being no method handlers for '[]' on the object which is very odd, as I have validation expressions outside the object. If someone could give a hand on why this is happening, that'd be great.
Implementation:
class MxMongo
-»»»# Initialise everything we need
-»»»def initialize(host : String, username : String, password : String, port : String)
-»»»-»»»@client = Nil
-»»»-»»»@database = "aggregation"
-»»»-»»»self.connect(host, username, password, port)
-»»»end
-»»»
-»»»# Connect to the database
-»»»def connect(host : String, username : String, password : String, port : String)
-»»»-»»»@client = Mongo::Client.new "mongodb://#{username}:#{password}@#{host}:#{port}/#{@database}"
-»»»-»»»return @client
-»»»end
-»»»# Get an instance of the database
-»»»def get_instance
-»»»-»»»if @client.not_nil! && db.not_nil!
-»»»-»»»-»»»return @client
-»»»-»»»else
-»»»-»»»-»»»return Nil
-»»»-»»»end
-»»»end
-»»»def get_collection(collection : String)
-»»»-»»»if [email protected]? && !collection.nil?
-»»»-»»»-»»»client = @client
-»»»-»»»-»»»return client.not_nil![collection] if collection.not_nil!
-»»»-»»»else
-»»»-»»»-»»»return Nil
-»»»-»»»end
-»»»end
end
Error I recieve:
test = mongo_client.get_collection("aggregation")
^~~~~~~~~~~~~~
in classes/init_mongo.cr:27: undefined method '[]' for Nil:Class (compile-time type is (Mongo::Client | Nil:Class))
return client.not_nil![collection] if collection.not_nil!
^
================================================================================
Nil:Class trace:
/usr/lib/crystal/object.cr:156
def not_nil!
^~~~~~~~
/usr/lib/crystal/object.cr:157
self
^~~~
Apologies for the formatting, copied from vim
Sure, there was a last commit some time ago, but there are so old issues and pull requets, also it isn't working with the current crystal version.
You solved on error, than you have another one. No end there.
There are a few minor syntax changes that prevent mongo.cr from compiling with the new version of Crystal. I fixed a few of them manually, however the last one is a legit bug that I have not been able to figure out, involving how the port number is parsed in certain circumstances. The spec failure I was unable to fix is this one:
crystal spec spec/uri_spec.cr:21 # Mongo::Uri should be able to create new uri with host and port
I'll post a PR if I do get it working
in lib/mongo/src/mongo/collection.cr:150: instantiating 'insert(JSON::Any, LibMongoC::InsertFlags, Nil)'
def insert(document, flags = LibMongoC::InsertFlags::NONE, write_concern = nil)
^
in lib/mongo/src/mongo/collection.cr:151: undefined method 'to_bson' for JSON::Any (did you mean 'to_json'?)
unless LibMongoC.collection_insert(self, flags, document.to_bson, write_concern, out error)
Below error when trying to retrieve value by key from a BSON object
Error in src/api.cr:8: instantiating 'Mongo::Collection#find(Hash(String, Hash(String, Int32)))'
collection.find({"age" => {"$gt" => 30}}) do |doc|
^~~~
in lib/mongo/src/mongo/collection.cr:133: instantiating 'Mongo::Cursor#each()'
find(query, fields, flags, skip, limit, batch_size, prefs).each do |doc|
^~~~
in lib/mongo/src/mongo/collection.cr:133: instantiating 'Mongo::Cursor#each()'
find(query, fields, flags, skip, limit, batch_size, prefs).each do |doc|
^~~~
in src/api.cr:8: instantiating 'Mongo::Collection#find(Hash(String, Hash(String, Int32)))'
collection.find({"age" => {"$gt" => 30}}) do |doc|
^~~~
in src/api.cr:10: instantiating 'BSON#[](String)'
puts doc["age"]
^
in lib/mongo/src/bson.cr:127: instantiating 'fetch(String)'
fetch(key) { raise IndexError.new }
^~~~~
in lib/mongo/src/bson.cr:116: instantiating 'BSON::Value#value()'
Value.new(value).value
^~~~~
in lib/mongo/src/bson/value.cr:42: undefined constant Time::Kind::Utc
Time.new(spec, Time::Kind::Utc)
Hello.
I try to run example from README.md and stuck on following error:
Error in ./src/limnal/controller.cr:1: while requiring "./mongo"
require "./mongo"
^
in ./libs/mongo/mongo.cr:2: while requiring "./bson"
require "./bson"
^
Syntax error in ./libs/mongo/bson.cr:63: expecting token ')', not 'as'
LibBSON.bson_free(cstr as Void*)
^
My code:
require "./mongo"
client = Mongo::Client.new "mongodb://<here I use mlab hosted database>"
db = client["limnal"]
collection = db["thread"]
collection.insert({"name" => "James Bond", "age" => 37})
collection.find({"age" => {"$gt" => 30}}) do |doc|
puts doc
end
Please help. thanx
Since Collection#insert returns nil, any idea how to find out what _id mongodb auto-added to the new document?
In ruby mongo driver to find all documents in collection is
collection.find
However, in this driver expect to have some arguments pass to it. Which argument should I pass to find
method in order to get the ideal result ?
> crystal spec
Error in line 1: while requiring "./spec/bulk_operation_spec.cr"
in spec/bulk_operation_spec.cr:1: while requiring "../src/mongo"
require "../src/mongo"
^
in src/mongo.cr:2: while requiring "./bson"
require "./bson"
^
in src/bson.cr:3: while requiring "./bson/*"
require "./bson/*"
^
in src/bson/value.cr:14: structs can't have finalizers because they are not tracked by the GC
def finalize
^~~~~~~~
From https://github.com/crystal-lang/crystal/blob/master/CHANGELOG.md#0204-06-01-2017
(breaking change) Defining a finalize method on a struct now gives a compile error
Because of memory leaks, see crystal-lang/crystal#3840
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.