kirilltitov / fdbswift Goto Github PK

We need to make sure that everything is working correctly with recently released major version of database. Hopefully, no changes and version bump needed

Stdlib precondition fails silently in release builds, which is nonsense. We should migrate all precondition calls to a new function

public func _precondition(
    _ condition: @autoclosure () -> Bool,
    _ message: @autoclosure () -> String = String(),
    file: StaticString = #file, line: UInt = #line
) {
    guard condition() == true else {
        fatalError("Precondition failed: \(message())", file: file, line: line)
    }
}

The readme and the code are different. Below is code copied from the readme and line 370 gets an error. I am running 4.2.0.

Is there any interest in adding support for 80-bit and 96-bit Versionstamps to Tuple? The .setVersionstampedKey option is already there, so all that would need to be done is to add a type code for version stamps, and a check in Transaction.atomic(_:key:value:) to look for a single Versionstamp type, and append it's position within the key to the end of the key's byte string.

If you are fine with the addition, I'd be more than happy to submit a PR for this, since I've found a need for it.

https://travis-ci.org/github/kirilltitov/FDBSwift/builds/725863989#L719

cc @dimitribouniol

That's an oof, because I don't really understand how to port it from Python or Go, because it utilizes [blocking, obviously] mutexes heavily, and it's not quite clear

1. Why use them
2. How to use them [in Swift-NIO context]

Appendix:

1. High-level documentation: https://apple.github.io/foundationdb/developer-guide.html#directories
2. Python implementation: https://github.com/apple/foundationdb/blob/master/bindings/python/fdb/directory_impl.py
3. Go implementation: https://github.com/apple/foundationdb/blob/master/bindings/go/src/fdb/directory/allocator.go

I'm trying to play around with FDB in a simple Vapor project, but I'm having trouble getting started. Specifically, when building with Xcode, I get the following errors:

<module-includes>:1:9: note: in file included from <module-includes>:1:
#import "shim.h"
        ^
/Users/dimitribouniol/Library/Developer/Xcode/DerivedData/hello-bfiigegfzdngdvepisiicspgaxaf/SourcePackages/checkouts/FDBSwift/Sources/CFDB/shim.h:5:14: error: 'foundationdb/fdb_c.h' file not found
    #include "foundationdb/fdb_c.h"
             ^
/Users/dimitribouniol/Library/Developer/Xcode/DerivedData/hello-bfiigegfzdngdvepisiicspgaxaf/SourcePackages/checkouts/FDBSwift/Sources/FDB/FDB.swift:3:8: error: could not build Objective-C module 'CFDB'
import CFDB
       ^

This is my Package.swift:

// swift-tools-version:5.2
import PackageDescription

let package = Package(
    name: "hello",
    platforms: [
       .macOS(.v10_15)
    ],
    dependencies: [
        // 💧 A server-side Swift web framework.
        .package(url: "https://github.com/vapor/vapor.git", from: "4.0.0-rc"),
        .package(url: "https://github.com/kirilltitov/FDBSwift.git", from: "4.0.0")
    ],
    targets: [
        .target(
            name: "App",
            dependencies: [
                .product(name: "Vapor", package: "vapor"),
                .product(name: "FDB", package: "FDBSwift")
            ],
            swiftSettings: [
                // Enable better optimizations when building in Release configuration. Despite the use of
                // the `.unsafeFlags` construct required by SwiftPM, this flag is recommended for Release
                // builds. See <https://github.com/swift-server/guides#building-for-production> for details.
                .unsafeFlags(["-cross-module-optimization"], .when(configuration: .release))
            ]
        ),
        .target(name: "Run", dependencies: [.target(name: "App")]),
        .testTarget(name: "AppTests", dependencies: [
            .target(name: "App"),
            .product(name: "XCTVapor", package: "vapor"),
        ])
    ]
)

Is there something obvious that I'm missing? Things seem to build via swift build (the process of which I'm fairly unfamiliar with), but not in Xcode. I did make sure to install the pkgconfig:

Welp, recently FoundationDB released an official alpha version of FDB connector https://github.com/foundationdb/fdb-swift-bindings.

So we have an open question: can FDBSwift compete with official adapter or not? Can it be better or offer unique/handy functionality?

I just noticed there there is no license for this repo. Could one be added?

Should not cause runtime error if invalid bytes are passed, instead gentle error should be thrown.

let packed: Bytes = Bytes([0, 0, 0]).pack()
dump(Tuple(from: bytes))

fails with array out of bounds error

Currently FDB extends builtin types like Bool, Int etc with method pack() (and _pack()) as per FDBTuplePackable protocol.

This naming is too wide, whereas it should be FDB-specific. Proposed naming is getPackedFDBTupleValue (and a respective underscored method).

Now that I think of it, do we really need these methods as methods? Can't we just stick with [computed] vars? Methods aren't throwing, and I really can't recall why did I define the protocol the way I defined it. Vars can still be inlinable...

This change should land either in v5, or in v4 with deprecating (obsoleting?) existing implementation.

One thing that bothers me: this repo isn't searchable in Google by "foundationdb swift" query.

Literally every other search engine shows this repository correctly (at least on first page, ideally on first position) — Yandex, Bing, DuckDuckGo.

But not Google (still it's in index and available by DIRECT url search)

What is going on? :(

I'm not sure if anyone else (👋 @kirilltitov) has run into this, but I cannot run tests when the package is built using Xcode 12. The following errors are reported, indicating the test target cannot find libfdb_c.dylib:

2020-10-08 12:43:14.476787-0700 xctest[82265:4451631] The bundle “FDBTests” couldn’t be loaded because it is damaged or missing necessary resources. Try reinstalling the bundle.
2020-10-08 12:43:14.476968-0700 xctest[82265:4451631] (dlopen_preflight(/Users/dimitribouniol/Library/Developer/Xcode/DerivedData/FDBSwift-avmdmjjucugpndfvczrsqjfgiexd/Build/Products/Debug/FDBTests.xctest/Contents/MacOS/FDBTests): Library not loaded: libfdb_c.dylib
  Referenced from: /Users/dimitribouniol/Library/Developer/Xcode/DerivedData/FDBSwift-avmdmjjucugpndfvczrsqjfgiexd/Build/Products/Debug/FDBTests.xctest/Contents/MacOS/FDBTests
  Reason: image not found)
Program ended with exit code: 82

This extends to packages that depend on FDBSwift. Running the tests via SwiftPM (Swift 5.3) directly runs the tests fine. Building and running an app that depends on FDBSwift via Xcode also works fine.

I initially though this could have something to do with systemLibrary support in SwiftPM, so I made a bare-bones package that imported libfdb, and saw the same issues. However, switching the library to libevent (A random pkgConfig I had installed on my system) built and ran the tests fine, so I think it may be specific to the fdb pkgConfig?

Here is the libevent.pc:

#libevent pkg-config source file

prefix=/usr/local/Cellar/libevent/2.1.11_1
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
includedir=${prefix}/include

Name: libevent
Description: libevent is an asynchronous notification event loop library
Version: 2.1.11-stable
Requires:
Conflicts:
Libs: -L${libdir} -levent
Libs.private: 
Cflags: -I${includedir}

Compared to my libfdb.pc:

prefix=/usr/local
exec_prefix=${prefix}
includedir=${prefix}/include
libdir=${exec_prefix}/lib

Name: fdb
Description: FoundationDB macOS library
Version: 6.2.22
Cflags: -I${includedir}
Libs: -L${libdir} -lfdb_c

I'd appreciate any pointers if it seems I'm doing something wrong, or if this is reproducible at all...

I am trying transactions using the simple set (no future) from Sources/FDB/Transaction/Transaction+Sync.swift line 14 that says
func set(key: AnyFDBKey, value: Bytes, commit: Bool) throws {
but I am getting an Ambiguous use of 'set(key:value:commit:)' because Sources/FDB/Transaction/Transaction+NIO.swift line 45 has the same method signature with a different result
func set(key: AnyFDBKey, value: Bytes, commit: Bool) -> EventLoopFuture<AnyFDBTransaction> {

The tests that are in the repository import FDB in as @testable and uses internal methods, so they are no help. Any help would be valuable.

testStringKeys() and testStaticStringKeys() both add a key/value foo to the local cluster, but don't clean them up. Should these have been in the test's subspace instead?

Verbose mode should be really verbose (additional attention to connection process). Probably adopt official Swift Server Logger.

The official swift bindings seem to use an additional package (https://github.com/FoundationDB/fdb-swift-c-packaging) to handle importing the C API on various platforms — although I understand not using the official repos (they haven't been updated for 2 years, and don't have tagged releases for specific FDB versions), is there a reason this library doesn't use the same approach? I imagine it would make maintaining future versions and re-installation on new systems a lot easier.

Hi,

thanks for the heads up on apple/foundationdb#204. I see you already have the async interface on your ToDo list. I would recommend to primarily focus on the async interface. For synchronous access a simple wait() function should be fine.

Swift NIO has a great concept for asynchronous futures and the first web-frameworks adapt this concept (e.g. Vapor 3).

Would it be possible to have Swift NIO style Futures and Promises?

Thanks!

As parent call always do-catches all inner calls, it's obsolete.

Future+KeyValue.swift, Future+Bytes.swift.

Everything should start from FDB: FDB.Transaction, FDB.Tuple etc. Exceptions: protocols (damn it) — TuplePackable >> FDBTuplePackable until Swift allows proper nested types or enables proper namespaces.

Because it's always a string :) It will require a major version bump to actually rename it, but for now we can add a static proxy method.