I was able to get the library working on iOS, however, I am using it only for parsing. I took an existing JSON document and converted to BSON using the bsontools. When I load that document into the parser, I only get a single record out even though the JSON had an array of 408 records. I may just be using it wrong, but here's my code:

import UIKit

class ViewController: UIViewController {

    @IBOutlet weak var textView: UITextView!

    override func viewDidLoad() {
        super.viewDidLoad()

        if let resource = Bundle.main.url(forResource: "categories", withExtension: "bson") {
            if let data = try? Data(contentsOf: resource) {
                let doc = Document(data: data)
                textView.text = "\(doc)"
            }
        }
    }
}

Here's what the output looks like in a text view:

and I've attached the BSON and corresponding JSON file for reference:

categories.zip

I haven't look into this bug further, but it was not there in the past. I am getting the following:

I hope the screenshots help

Number like 19,21,666... and so on won't be converted back.
BSON Int limit: 9223372036854775807
so every number lower then 9223372036854775807 should be able to be converted back.

https://github.com/OpenKitten/BSON/blob/3edad325da31095c9d54d04c6125f1fec1f3c5bf/Sources/BSON/Codable.swift#L660

For: 9223372036854775807 is correct right?

fatal error: 'try!' expression unexpectedly raised an error: Swift.EncodingError.invalidValue(9223372036854775807, Swift.EncodingError.Context(codingPath: [], debugDescription: "Value exceeds 9223372036854775807 which is the BSON Int limit"

• Improve the Readme
• Complete the documentation
• Complete the unit tests
• Re-evaluate implementations

Tried upgrading to BSON 6 and received lots of compiler messages. Looks like a lot of objects have been removed or just changed to how they are used. Is there any documentation on how best to update your BSON 5 code to work in BSON 6?

Hi, while I am decoding my Document, I can see that BSONDecoder doesn't treat my snake case key like standard swift JSONDecoder with option convertFromSnakeCase in KeyCodingStrategy. I have to explicit use CodingKey to map snake case to camel case very handy. Also, I go to BSONDecoder implementation and it seems like don't have any options like convertFromSnakeCase. As my question, BSON will handle this option or something which will handle this?

Thanks!

I wasn't sure where to put this, so I thought I'd put it here. Swift 4 will offer a new Codable protocol with which you can automagically encode and decode classes and structs to and from data structures such as json. So bson should work too of course, only there's not encode / decoder for that yet.

I would love it if you'd be willing to write a BSON encoder / decoder that works with the codable protocol in Swift 4. Would you be willing to?

Hello Jiannis,

This week I am porting everything to Swift 4.0 and I updated BSON from 5.0 to 5.1

I was getting weird behaviour on with database queries, object creation etc. After some debugging I managed to pin the problem down. The following lines behave differently in 5.0 and 5.1. The behaviour I am getting in 5.1, i do not believe it is the expected one.

    var document = Document()
    document.removeValue(forKey: "_id")
    document.setObject(object: "123", key: "_id")
    print(document.dictionaryRepresentation["_id"]) // prints "nil" used to print "Optional("123")"
    document.removeValue(forKey: "_id")
    document.setObject(object: "456", key: "_id")
    print(document.dictionaryRepresentation["_id"]) // prints "nil" used to print "Optional("456")"
    document.setObject(object: "789", key: "_id")
    print(document.dictionaryRepresentation["_id"]) // prints "nil" used to print "Optional("789")"
    document.setObject(object: "anyvalue", key: "anykey")
    print(document.dictionaryRepresentation["anykey"]) // prints "nil" used to print "Optional("anyvalue")"

    var document2 = Document()
    document2.removeValue(forKey: "anykey")
    document2.setObject(object: "123", key: "_id")
    print(document2.dictionaryRepresentation["_id"]) // prints "nil" used to print "Optional("123")"

    var document3 = Document()
    document3.setObject(object: "123", key: "_id")
    print(document3.dictionaryRepresentation["_id"]) // prints "Optional("123")"
    document2.removeValue(forKey: "anykey")
    print(document3.dictionaryRepresentation["_id"]) // prints "Optional("123")"
    print(document3.dictionaryRepresentation["anykey"]) // prints "nil"

Basically what is happening is that if on a fresh document (have not set any keys yet), I try to remove any key, then the document gets in a state that I cannot set any keys.

If we init a document and set any key. Then removing a key that does not exist, will not cause any problems. It only causes problem if the document is fresh.

For the moment I sorted it out by implementing the following function and using it instead of removeValue(forKey:)

extension Document {
    
    public mutating func removeValueIf(forKey key: String) {
        guard self.dictionaryRepresentation[key] != nil else {
            return
        }
        self.removeValue(forKey: key)
    }
}

I was working with MongoKitten today and found a weird and unexpected behavior i.e.., the existence of duplicate keys.
Please refer the image below for better clearity.

What I had tried to do and noticed is as follows -

let gotFromDb = try collection.find(["email": eMail_])
var user = try gotFromDb.find().first
Till here 👆 is the same code(or common code), and after that I am getting different results based on different and those are the things I want to share here.

If I do the following -
user.append("Mitesh", forKey: "something")
user["something"] = ["ABCD", "eFgH"]
try collection.update(["email": _eMail], to: user, upserting: false, multiple: false, writeConcern: nil, stoppingOnError: true)
I face the issue of getting the duplicate keys.

Next case, If I do this -
let myName: Document = ["myName": "\(arc4random_uniform(UInt32(999)))"]
user.append(contentsOf: myName) -> This will not create an entry with duplicate key instead it will update to the new value.
user.append(myName) -> This will create an entry with duplicate key
try collection.update(["email": _eMail], to: user, upserting: false, multiple: false, writeConcern: nil, stoppingOnError: true)

Next case, If I do this -
let tokenDocument: Document = ["lalla": ["\(arc4random_uniform(UInt32(999)))", "XKCD", "somethign", "Ron"]]
user.append(contentsOf: tokenDocument)
try collection.update(["email": _eMail], to: user, upserting: false, multiple: false, writeConcern: nil, stoppingOnError: true)
It works properly I mean the new value overrites the previous one and no value with duplicate key is added.

Currently I need to convert from Foundation.Data to [UInt8] to insert some data into my BSON document. However, I need to allocate a ~300 kiB [UInt8] buffer to store the data, and my web app freezes. Therefore I'm wondering if maybe the native Data type should be used to avoid this transformation, which I already see here.

There is a Sourcery template for generating LinuxMain.swift.

class A: Codable {
var a: String
}
class B: A {
var b: String
}
if one properly do implement CodingKeys enum and encode and decode in the subclass, the Foundation JSONDecoder/Encoder (as well as other third-party decoders such as XMLCoder, CodableCSV, ...) do correctly encode/decode properties of both class A and class B if one encode/decode a B object. However, BSON does not seem to be working in this scenario. It seems to not call/use the parent class container.

The README says to use the Swift package manager, but it's unclear as to how to do so. Running the command swift build from the command line finishes without an error, but now what? What can I do with the build product. If it's just importing the frameworks, that's fine, but next steps are unclear. Where do I get the .framework file?

Can I simply add the sources to my project and build that way? That would be my preference. I've tried that, but I get the error:

BSON/Sources/ValueConvertible.swift:89:11: Use of undeclared type 'RegularExpression'

Thanks in advance.

Document+Mutations.swift:6:17: 'changeCapacity(to:)' is deprecated: changeCapacity has been replaced by reserveCapacity

There seems to be a weird issue encoding UInts. Consider the following example:

// Code
import BSON

do {
  struct Foo: Codable {
    var record: UInt64

    init(record: UInt64) {
      self.record = record
    }
  }

  let firstFoo = Foo(record: 283467841569)
  let document = try BSONEncoder().encode(firstFoo)
  let secondFoo = try BSONDecoder().decode(Foo.self, from: document)

  print("Decoded: \(secondFoo.record)")
} catch {
  print("Error: \(error)")
}

// Output:
Error: dataCorrupted(Swift.DecodingError.Context(codingPath: [CodingKeys(stringValue: "record", intValue: nil)], debugDescription: "BSON number <283467841569> does not fit in UInt", underlyingError: nil))

Running swift:

Welcome to Apple Swift version 4.1.2 (swiftlang-902.0.54 clang-902.0.39.2). Type :help for assistance.
  1> print(UInt.max)
18446744073709551615

283467841569 should fit fine in a UInt.

We should look at areas where performance can be improved. Let's be faster than C libbson!

• Change to arrayslice where applicable

• Go
• Dart
• C
• Swift/C (Danapple)
• Ruby
• PHP

Still have to set the Swift version to 3.2 when running in Xcode10 and Swift 4.2. Have a warning that Swift 3 mode is deprecated. I tried the latest version of 5.2.4 without the Swift 3 mode, and it fails to compile. Attached a screen snippet of the compiler error.

If a number of instance of an app are started in parallel there is a danger that if they start in the same second there can be duplicate objectIds generated.

Perhaps time in millisecs could be used.

Or is there a source of true random on linux (/dev/random)

#if os(Linux)
private static var random: Int32 = {
srand(UInt32(time(nil)))
return rand()
}()
private static var counter: Int32 = {
srand(UInt32(time(nil)))
return rand()
}()
#else

I tried to use this package with MongoKitten on a big endian machine. At that time, I had endian problems. For the master/5.2 branch, I needed some fixes like as follows. Although I plan to make PRs to support big endian, I am wondering to know which branch should be used as the code base. Because, it seems that this package is being re-written in the develop/6.0/rewrite branch.

+#if _endian(little)
                 let double: Double = try fromBytes(storage[position..<position+8])
+#else
+                var tmp = Bytes(repeating: 0, count: 8)
+                var j: Int = 0
+                for i in (position ..< position + 8).reversed() {
+                    tmp[j] = storage[i]
+		    j+=1
+		}
+                let double: Double = try fromBytes(tmp)
+#endif

See https://gist.github.com/Obbut/d7b465203c16399e2d2b8901aebea4b3

If a document's subdocument gets extracted, the original document gets this subdocument mutated and the subdocument copy is read. What happens to the cache?

Encoding a container of objects which don't include any Codable attributes fails. JSONEncoder is able to handle this use case. A test demonstrating the issue is provided below. While the test is clearly a contrived use case which is unlikely to occur in practice, it is probable that this issue could occur in a polymorphic design where the one of the polymorphic subclasses does not include any Codable attributes (i.e. no persistent attributes). However, I have not tested the polymorphic use case and don't have an immediate need for it. This issue is not blocking my use of OpenKitten/BSON.

    func testEmptyCodable() {
        
        class EmptyCodable : Codable {}
        let encoder = BSONEncoder()
        let decoder = BSONDecoder()
        var document = Document()
        let emptySource = EmptyCodable()
        do {
            document = try encoder.encode(emptySource)
        } catch {
            XCTFail ("Expected success but got \(error)")
        }
        do {
            let _ = try decoder.decode(EmptyCodable.self, from: document)
        } catch {
            XCTFail ("Expected success \(error)")
        }
        let arrayOfEmptySource = [EmptyCodable(), EmptyCodable()]
        do {
            document = try encoder.encode(arrayOfEmptySource)
        } catch {
            XCTFail ("Expected success but got \(error)")
        }
        // JSONEncoder and JSONDecoder can handle [EmptyCodable]
        do {
            let jsonEncoder = JSONEncoder()
            let data = try jsonEncoder.encode(arrayOfEmptySource)
            let jsonDecoder = JSONDecoder()
            let decodedArray = try jsonDecoder.decode ([EmptyCodable].self, from: data)
            XCTAssertEqual (2, decodedArray.count)
        } catch {
            XCTFail ("Expected success but got \(error)")
        }
    }

No "instantiate" for a factory.. something like "makeElement" would fit better

Will this work with Xcode 7.3.1* which uses Swift 2.2.1?

The snapshot version makes xCode crash for me so I'm going to be using the stable release rather than the snapshot.
Not sure if this will cause issues.

I was trying to persist an Int value and extract it from Document, but the extraction always fails when used with generic extract<Int> function.

let intValue: Int = 300
var document: BSON.Document = [ "value" : intValue.makeBsonValue() ]
guard let extractedIntValue: Int = document.extract("value") else {
    fatalError("Failed to extract Int.")
}

by changing it from platform independent Int to any of Int32 or Int64 the extraction works fine.

extension Optional where Wrapped == Primitive {
^
/Users/Mac/Today-News-Admin/Packages/BSON-5.1.4/Sources/BSON/Interpreting.swift:204:32: error: same-type requirement makes generic parameter 'Key' non-generic
extension Dictionary where Key == String, Value == Primitive {
^
/Users/Mac/Today-News-Admin/Packages/BSON-5.1.4/Sources/BSON/Interpreting.swift:204:49: error: same-type requirement makes generic parameter 'Value' non-generic
extension Dictionary where Key == String, Value == Primitive {
^
/Users/Mac/Today-News-Admin/Packages/BSON-5.1.4/Sources/BSON/Interpreting.swift:194:31: error: same-type requirement makes generic parameter 'Element' non-generic
extension Array where Element == Primitive {
^
/Users/Mac/Today-News-Admin/Packages/BSON-5.1.4/Sources/BSON/Document.swift:14:31: error: same-type requirement makes generic parameter 'Element' non-generic
extension Array where Element == Document {
^
/Users/Mac/Today-News-Admin/Packages/BSON-5.1.4/Sources/BSON/Optional+BSON.swift:11:34: error: same-type requirement makes generic parameter 'Wrapped' non-generic
extension Optional where Wrapped == Primitive {
^
/Users/Mac/Today-News-Admin/Packages/BSON-5.1.4/Sources/BSON/Interpreting.swift:204:32: error: same-type requirement makes generic parameter 'Key' non-generic
extension Dictionary where Key == String, Value == Primitive {
^
/Users/Mac/Today-News-Admin/Packages/BSON-5.1.4/Sources/BSON/Interpreting.swift:204:49: error: same-type requirement makes generic parameter 'Value' non-generic
extension Dictionary where Key == String, Value == Primitive {
^
:0: error: build had 1 command failures

Codable haven an issue while unwraping back value from document.

If I have an object with wariable
`var salt: [UInt8] = [0,1,2,3,4,5,68,122,254,255]

on getting back object with this error is returned

Initializing from document failed: dataCorrupted(Swift.DecodingError.Context(codingPath: [], debugDescription: "BSON number <0> does not fit in UInt8", underlyingError: nil))
Could not initialize User from document

What I've notice the range in Codable.swift is wrong.

instead of unwraping full range back into Uint8
0,1,2…253,254,255
This
https://github.com/OpenKitten/BSON/blob/0a0dd33ebe9b25c5c8ee488fe86dbb4c4fa48dd2/Sources/BSON/Codable.swift#L626
Constraints range to this 0 {1,2…253,254} 255 throwing an error when ever 0 or 255 appear.

That's true not only for UInt8 but others as well.

My example Model:

final class User: Model {
	var _id = ObjectId()
    var salt: [UInt8] //Temporary solution - store salt as hexString
    var password: String
    var login: String
}

Looks like the 5.2.5 tag was done on master/6.0 rather than master/5.2.

Big endian platforms will not like this

I am trying to use this package on a new project, and I'm unable to access any values on a created Document while following the same method as the README states.

The compiler is unable to use the value:

let data: Document = [ "type": "createSession" ]
let type = String(data["type"])
//         ^ No exact matches in call to initializer 
print(type);

If I add a breakpoint and observe the value using lldb, it does have a debugger output, so the data exists:

(lldb) vo data
(BSON.Document) data = ▿ ["type": "createSession"]
  ▿ storage : ByteBuffer (...)

Did this stop working on newer versions of Swift?

We should add a feature to initialize Documents with a Swift dictionary like [String : Any] or [String : AnyObject].

Hello:

I saw the new BSON 5.0 included some changes and I want to discuss this one:

Previously in BSON 4.0 I could create an enum with the string keys for my MongoDB collection objects and pass the enum as a parameter by making it extend the StringVariant protocol:

This provides safety against typo errors and a clearer syntax.

Is there any possibility to have it back?
If not, could you make the key of the dictionary be a protocol type (so we can extend it) instead of a class or struct type (like String)?

Thanks

The JSONEncoder serializes the following struct and dictionary literal equally,

struct TestDate {
    let date = Date()
}
let dictionary = [ "date": Date() ]

But the BSONEncoder, while encoding the TestDate correctly, failed to encode the values in dictionary as Date.

Reproduction

let dictionary = [ "date": Date() ]
var encoder = BSONEncoder()
let result = (try? encoder.encode(dictionary))!
print(result.dictionaryRepresentation["date"]!)

The result will print a Double value like 528704067.395462, this value is Date().timeIntervalSinceReferenceDate. It's reproducible on macOS High Sierra with Xcode 9 and default toolchain.

Analysis

The dictionary will call superEncoder(forKey:), which in turns call nestedEncoder(forKey:),
a single value container is requested, and the Double value is written. It's unknown when the conversion to Double happen, most likely in Swift Library.

How about you make Value also implement ValueConvertible:

extension Value : ValueConvertible {
    public func makeBsonValue() -> Value {
        return self
    }
}

And then you change BSON/Sources/Document+LiteralConvertibles.swift so that it accepts a Dictionary<String, ValueConvertible> instead of a Dictionary<String, Value>.

extension Document : ArrayLiteralConvertible {
    /// Initialize a Document using an array of `BSONElement`s.
    public init(array: [ValueConvertible]) {
        elements = array.map { ("", $0.makeBsonValue()) }
        self.enforceArray()
    }

    /// For now.. only accept BSONElement
    public init(arrayLiteral arrayElements: ValueConvertible...) {
        self.init(array: arrayElements)
    }

    public mutating func enforceArray() {
        for i in 0..<elements.count {
            elements[i].0 = "\(i)"
        }
    }
}

extension Document : DictionaryLiteralConvertible {
    public init(dictionaryElements: [(String, ValueConvertible)]) {
        self.elements = dictionaryElements.map { el in (el.0, el.1.makeBsonValue()) }
    }

    /// Create an instance initialized with `elements`.
    public init(dictionaryLiteral dictionaryElements: (String, ValueConvertible)...) {
        self.init(dictionaryElements: dictionaryElements)
    }
}

Then you don't need to ~ operator anymore for putting variables in a document. You could even go further and change self.elements to be of type [(String, ValueConvertible)] instead of [(String, Value)]. But I don't know how many consequences that has for your code...

What do you think?

In BSON 7.0 on the master branch, I've saved a document in MongoDB and copied out the raw string from Compass for testing etc.

I try to initialize the ObjectID from the raw string and get a crash

It will be useful to have a base protocol for BSONErrors that can be handled/caught at a global level for some applications. Currently if we want to catch a BSON error we would have to type cast to all the known types of errors. It would turn code that looks like this:

func handle(error: Error) {
    if let error = error as? BSONValueNotFound {
    ...
    } else if let error = error as? DeserializationError {
    ...
    } else if let error = error as? UnsupportedDocumentDecoding {
    ...
    }
    etc..
}

To code that would look like this:

if let error = error as? BSONError {
...
}

BSON doesn't compile anymore in Swift 5.5 from Xcode 13 beta 4 due to a fix for SR-14848. From the release notes of Xcode 13 beta 4:

The Swift standard library no longer provides a nonfunctional default implementation of the subscript(bounds: Range) accessor for Collection implementors whose SubSequence isn’t Slice. Define your code’s subscript(bounds: Range) -> SubSequence, or use Slice as your Collection’s SubSequence. (SR-14848, 79891982)

Errors are:

type 'Document' does not conform to protocol 'RandomAccessCollection'
type 'DocumentSlice' does not conform to protocol 'RandomAccessCollection'

When trying to build, I always receive the error <unknown>:0: error: cannot load module 'BSON' as 'bson'.

Could one kindly assist? Any help much appreciated!

• Basic usage
• Extended JSON
• Implementation in projects

I use BSON in MongoKitten and I have a aggregation, that returns two Document. The query itself is very fast, it just needs 0.009 seconds. But printing out the documents increases the time to 0.22 seconds!

This is my code:

let start = DispatchTime.now()
		
let pipeline = benutzerCollection.aggregate([
    .lookup(from: "userGroups", localField: "userGroupID", foreignField: "_id", as: "group"),
    .unwind(fieldPath: "$group"),
    match("group.hasMagic" == true)
])
		
pipeline.execute().whenComplete { result in
    switch result {
    case .success(let finalizedCursor):
        finalizedCursor.cursor.forEach { document in
		print(document)
	}
				
	let end = DispatchTime.now()
				
	let nanoTime = end.uptimeNanoseconds - start.uptimeNanoseconds
	let timeInterval = Double(nanoTime) / 1_000_000_000 // Technically could overflow for long running tests
				
	print("Time to evaluate: \(timeInterval) seconds")
...

I run the example and took a look at Instruments. Please have a look:

Is there any way how to improve this performance? Maybe I have to tell, that the documents are not very small and have a lot of key-value-properties (mostly boolean). But running the same query for example in Robo3T immediatly prints out the JSON string.