weebly / cereal Goto Github PK

I’m currently working with Swift 3.0. Eventual goal is port apps from Linux to OS X.

Swift 4 introduces new Codable protocol that generally do the same as this library.
Should we make a migration tool for those who want to use a new protocol and be able to keep backward compatibility with Cereal?

Hi! I've got a fun, maybe simple question for you about using Cereal with an enum:

I've got a small class and an enum, which look like:

enum UploadEnum {
    case Sending
    case Sent
    case Failed(error: NSError)
}

class DataClass {
    let name: String
    let upload: UploadEnum
}

I want to encode DataClass with Cereal, but… I don't know how, because of UploadEnum.

With other enums without associated types, I've encoded the enum's rawValue, and then decoded it with the enum's .init(rawValue: <>) initializer.

Can you think of any simple ways to encode UploadEnum? I can think of some invasive approaches which require changing the enum, but I'm seeking alternatives.

Hey, I'd love to see this lib available in Swift 3 so I can implement it in my project.

Thanks 😄

Hi, I'm currently using Cereal 1.4.0 in a Swift 3 project on OS X ( I created an OS X target, which has been working fine for me so far, with the exception of this issue).

I'm not sure if my recent upgrade to Swift 3 and Cereal 1.4.0 introduced this issue, or not, because I'm sure it worked previously but not sure exactly when it stopped working.

Anyway, I'm serializing an array of Swift objects, in this case the array includes only one object.

The error I'm getting is ".Array / .Cereal / .Dictionary not expected", so of course at first I thought the problem was that I was deserializing an array (even though that should be supported). However on closer inspection, the issue is with the decoding of a property in the object which is a dictionary. parseEncodedDictionaryString is being called, which calls iterateEncodedValues, which eventually calls parseEncodedDictionaryString. This then fails decoding the key, apparently because the key's type is .cereal.

Here's all the data that's failing in parseEncodedDictionaryString:

keyType = CerealOSX.CerealTypeIdentifier.cereal
keyValue = "k,8:rawValue:s,4:more"
type = CerealOSX.CerealTypeIdentifier.string
value = "100"

Any idea why this is failing? Also what does the key type of .cereal mean and why is that unexpected? I'm getting no errors during the encoding/serialization, only in decoding. Is this a bug in Cereal, or something I'm doing wrong?

Any help is greatly appreciated, thanks!

Unclear whether this is due to a bug in Cereal of not, but encoding an NSDate and then decoding it seems to result in a slightly less accurate date, which then in turn causes isEqualToDate() to return false for the two dates. Here's a minimal test case to highlight the problem:

let date = NSDate()
let data = try! CerealEncoder.dataWithRootItem(date)
let comparison: NSDate = try! CerealDecoder.rootItemWithData(data)
XCTAssertTrue(date.isEqualToDate(comparison))

The final line is not true, while logically it should be. When logging out info about the dates, they seem correct, but there is actually a problem with the sub-second part of the dates, Cereal seems to be dropping some accuracy:

Original: 471798680.624343 After encode/decode: 471798680.62434

I'm using Cereal to wire up some simple Swift struct serialization and deserialization to disk (in UserDefaults).

Before I write the objects to disk, I began testing serialization in-memory, and am getting a weird failure from Cereal:

// deviceRecordIn is a Swift struct which conforms to `CerealType`

        // When:
        // We serialize one in-memory.
        let encodingKey = "deviceRecord"
        var encoder = CerealEncoder()
        do {
            try encoder.encode(deviceRecordIn, forKey: encodingKey)
        } catch {
            XCTAssertTrue(false, "Error encoding data. \(error)")
        }
        let data = encoder.toData()
        XCTAssertNotNil(data)

        // Then:
        // We should be able to de-serialize it:
        do {
            let decoder = try CerealDecoder(data: data)
            let aRecord = try decoder.decode(encodingKey)

            // And it should match our serialized record.
            XCTAssertNotNil(aRecord)
            guard let encodedRecord = aRecord else {
                return
            }
        } catch {
            XCTAssertTrue(false, "Error encoding data. \(error)")
        }

My unit test fails at let aRecord = try decoder.decode(encodingKey), with:

error: -[InspireTests.BluetoothPairingTest testThatWeCanSerializeAPairingRecord] : XCTAssertTrue failed - Error encoding data. InvalidEncoding("Failed to instantiate CerealTypeIdentifier with k")

The call stack (sorry, this one is pretty verbose):

(lldb) bt
* Thread 1 (tid 0x43e9f0, queue "com.apple.main-thread"): 0x0000000115dade40 swift_willThrow
    Stopped by breakpoint 27.1
    #0  0x0000000115dade40 swift_willThrow
  * #1  0x000000011573720e static CerealDecoder.(encodedString="k,4:uuid:s,36:16B6F74C-8336-48A0-B833-F920C367D0CF:k,9:sensorUID:i,1:0:k,14:connectionDate:T,16:477003543.503419", index=Swift.String.Index @ 0x00007fff550b24c0, $error=ErrorType @ 0x00007fff550b2e60)(String, startingAtIndex : String.CharacterView.Index) throws -> (type : CerealTypeIdentifier, indexPassedValue : String.CharacterView.Index) + 1694 at CerealDecoder.swift:1149
    #2  0x0000000115738341 static CerealDecoder.(encodedString="k,4:uuid:s,36:16B6F74C-8336-48A0-B833-F920C367D0CF:k,9:sensorUID:i,1:0:k,14:connectionDate:T,16:477003543.503419", index=Swift.String.Index @ 0x00007fff550b2780, $error=ErrorType @ 0x00007fff550b2e60)(String, startingAtIndex : String.CharacterView.Index) throws -> (type : CerealTypeIdentifier, value : String, endIndex : String.CharacterView.Index) + 289 at CerealDecoder.swift:1159
    #3  0x000000011572880c String.iterateEncodedValuesWithInstantationHandler(instantiationHandler=0x000000011573f260 Cereal`partial apply forwarder for static Cereal.CerealDecoder.((parseEncodedArrayString in _5CB6B52C5402593B67F608A3C7D8E4BB) (Swift.String) throws -> Swift.Array<Cereal.CerealRepresentable>).(closure #1) at CerealDecoder.swift, self="k,4:uuid:s,36:16B6F74C-8336-48A0-B833-F920C367D0CF:k,9:sensorUID:i,1:0:k,14:connectionDate:T,16:477003543.503419", $error=ErrorType @ 0x00007fff550b2e60) throws -> ()) throws -> () + 636 at CerealDecoder.swift:1313
    #4  0x00000001157276f5 static CerealDecoder.(encodedString="k,4:uuid:s,36:16B6F74C-8336-48A0-B833-F920C367D0CF:k,9:sensorUID:i,1:0:k,14:connectionDate:T,16:477003543.503419", $error=ErrorType @ 0x00007fff550b2e60)(String) throws -> [CerealRepresentable] + 181 at CerealDecoder.swift:1176
    #5  0x0000000115727563 CerealDecoder.decode(key="deviceRecord", self=Cereal.CerealDecoder @ 0x00007fff550b2c70, $error=ErrorType @ 0x00007fff550b2e60) throws -> [CerealRepresentable]? + 563 at CerealDecoder.swift:137
    #6  0x00000001156f6f54 BluetoothPairingTest.testThatWeCanSerializeAPairingRecord(self=0x00007fa9fa5a7b70) -> () + 884 at BluetoothPairingTest.swift:138
    #7  0x00000001156f7992 @objc BluetoothPairingTest.testThatWeCanSerializeAPairingRecord() -> () + 34 at BluetoothPairingTest.swift:0
    #8  0x000000010b5b989c __invoking___ + 140
    #9  0x000000010b5b96ee -[NSInvocation invoke] + 286
    #10  0x000000010abdb5a7 __24-[XCTestCase invokeTest]_block_invoke_2 + 362
    #11  0x000000010ac0fb2b -[XCTestContext performInScope:] + 190
    #12  0x000000010abdb42c -[XCTestCase invokeTest] + 169
    #13  0x000000010abdba56 -[XCTestCase performTest:] + 459
    #14  0x000000010abd94cb -[XCTestSuite performTest:] + 396
    #15  0x000000010abd94cb -[XCTestSuite performTest:] + 396
    #16  0x000000010abd94cb -[XCTestSuite performTest:] + 396
    #17  0x000000010abc62e4 __25-[XCTestDriver _runSuite]_block_invoke + 51
    #18  0x000000010abe71f4 -[XCTestObservationCenter _observeTestExecutionForBlock:] + 640
    #19  0x000000010abc6229 -[XCTestDriver _runSuite] + 453
    #20  0x000000010abc6fa5 -[XCTestDriver _checkForTestManager] + 259
    #21  0x000000010ac10fb2 _XCTestMain + 628
    #22  0x000000010ab4d20f ___lldb_unnamed_function3$$xctest + 362
    #23  0x000000010dfaf92d start + 1
    #24  0x000000010dfaf92d start + 1
(lldb) 

Interestingly, my struct's extension of init(decoder cereal: CerealDecoder) is not called as part of decoding.

Any pointers you can give to help me debug what's going wrong here?

1.2.1 does not work with Xcode 7.3 / Swift 2.2, but the master branch does. Could you release a 1.2.2 or 1.3?

i'm getting this error on decoding. Any idea why?

Set support is coming soon!

I have following struct

struct ItemDetail {
    var message:String?
    var date:NSDate?
    var location:String?
    var status:ItemStatus //enum

    init(message:String?, date:NSDate?, location:String?, status:ItemStatus) {
        self.message = message
        self.date = date
        self.location = location
        self.status = status
    }
}

However Cereal doesn't allow me to encode optionals. I don't want to check for nil or use default values. How can I use Cereal to encode/decode above struct? Thanks.

Suppose the following:

enum Gender: Int {
    case female
    case male
}

extension Gender: CerealType {
    // custom encode and decode
}

And encode was called on a variable of type Gender, then the RawRepresentable override version will be called, ignoring the custom serializers.

I know it's not a huge issue, but it could be fixed by providing an encodeCereal function for example, to explicitly invoke the custom serializers.

Carthage support would be nice

Hey,

i'm trying to encode a dictionary with variable value types. According to the Github Documentation one should cast the dictionary with .CER_casted() (since the value type is a protocol) before decoding/encoding. The decoding part works fine but for some reason i can't get the encoding to work.
The Dictionary is of the type:
[String: CerealType]

Its decoded like this:
guard let data:[String:CerealType] = try decoder.decode(dataKey)?.CER_casted() else { throw CodingError.MissingData } self.data = data

and encoded like this:

try encoder.encode(self.data.CER_casted(), forKey: dataKey);

which fails with:

Ambiguous Refrence to member 'encode(_:forKey:)

Any Ideas?

-- malte

I wrote an extension to serialize/deserialize dictionaries of cereal objects easily:

extension Dictionary where Key: CerealRepresentable, Value: CerealType {
    func serialize(to filePath: String) throws {
        var encoder = CerealEncoder()
        try encoder.encode(self, forKey: "dictionary")
        let data = encoder.toData()
        try data.writeToFile(filePath, options: [.AtomicWrite])
    }

    mutating func deserialize(from filePath: String) throws {
        guard let data = NSData(contentsOfFile: filePath) else { throw SerializationError.FileNotExists }

        let decoder = try CerealDecoder(data: data)
        guard let result: [Key: Value] = try decoder.decodeCereal("dictionary") else { throw SerializationError.IncorrectData }
        self = result
    }
}

And I faced with a very frustrating fact: when number of items to be serialized/deserialized counts in the thousands, Cereal consumes CPU a lot. For example, when I serialize 8k records, it takes about 5 seconds to run on my 6+ device:

When I replaced all string[index1..<index2] with string.substringWithRange(index1..<index2) + prepared dictionary's capacity (very empirical estimation: assuming number of commas equals number of subitems):

private static func parseEncodedCerealDictionaryString<DecodedKeyType: protocol<Hashable, CerealRepresentable>, DecodedValueType: CerealType>(encodedString: String) throws -> [DecodedKeyType: DecodedValueType] {
    let scanner = NSScanner(string: encodedString)
    scanner.charactersToBeSkipped = nil
    var scanResult: NSString?
    var approximateCapacity = 0
    while !scanner.atEnd {
        scanner.scanUpToString(",", intoString: &scanResult)
        approximateCapacity += 1
        scanner.scanString(",", intoString: nil)
    }
    var decodedItems = Dictionary<DecodedKeyType, DecodedValueType>(minimumCapacity: approximateCapacity)
    try encodedString.iterateEncodedValuesWithInstantationHandler { keyType, keyValue, type, value in
        let decodedKey: DecodedKeyType = try CerealDecoder.instantiate(keyValue, ofType: keyType)
        decodedItems[decodedKey] = try CerealDecoder.instantiateCereal(value, ofType: type) as DecodedValueType
    }

    return decodedItems
}

the picture changed a little bit, showing the real reason of high load:

There's too many CerealDecoder instantiations.
Should cereal move to another implementation, maybe the one that is based on NSScanner, with a very little string reallocations count, with a decoder reuse instead of reallocating.

Couldn't invent a way how to do that with backward compatibility with the existing format, maybe you can help me in pointing to the right way?

Right now thinking of making a transformer to NSKeyedArchiver/from NSKeyedUnarchiver since the items in CerealEncoder are stored as strings.