swhitty / flyingfox Goto Github PK
View Code? Open in Web Editor NEWLightweight, HTTP server written in Swift using async/await.
License: MIT License
Lightweight, HTTP server written in Swift using async/await.
License: MIT License
First of all, thanks for this really good library!
With this setup:
let server = HTTPServer(port: 8080)
let root = URL(fileURLWithPath: "/Users/myUser/myStaticDir")
let route = HTTPRoute(method: .GET, path: "/static/*")
let handler = DirectoryHTTPHandler(root: root, serverPath: "static")
await server.appendRoute(route, to: handler)
try await server.start()
When I perform:
curl -L -vv http://localhost:8080/static/index.html
I'm always getting a 404
.
That's because:
request.path
will always be /static/index.html
, so I find this example to be wrong: https://github.com/swhitty/FlyingFox#directoryhttphandler
To make it work I had to write the handler like this:
let handler = DirectoryHTTPHandler(root: root, serverPath: "/static/")
But then I found out that if I want to also handle the default file to be index.html
when not explicitly specifying it:
curl -L -vv http://localhost:8080/static/
or
curl -L -vv http://localhost:8080/static
I have to also add:
let fileRoute = HTTPRoute(method: .GET, path: "/static")
let fileHandler = FileHTTPHandler(path: root.appendingPathComponent("index.html"), contentType: "text/html")
await server.appendRoute(fileRoute, to: fileHandler)
Then I thought this is a very common scenario one would expect to work with minimal work, so I written this extension:
extension HTTPServer {
func appendStaticRoute(_ route: String, root: URL, defaultFileName: String = "index.html", withContentType contentType: String = "text/html") {
appendRoute(HTTPRoute(method: .GET, path: "/\(route)/*"), to: DirectoryHTTPHandler(root: root, serverPath: "/\(route)/"))
appendRoute(HTTPRoute(method: .GET, path: "/\(route)/"), to: FileHTTPHandler(path: root.appendingPathComponent(defaultFileName), contentType: contentType))
appendRoute(HTTPRoute(method: .GET, path: "/\(route)"), to: .redirect(to: "/\(route)/"))
}
func appendStaticRoute(_ route: String, bundle: Bundle = .main, defaultFileName: String = "index.html", withContentType contentType: String = "text/html") throws {
guard let root = bundle.resourceURL else { throw HTTPUnhandledError() }
appendStaticRoute(route, root: root, defaultFileName: defaultFileName, withContentType: contentType)
}
}
Which allows you to achieve the same with only:
let server = HTTPServer(port: 8080)
let root = URL(fileURLWithPath: "/Users/myUser/myStaticDir")
await server.appendStaticRoute("static", root: root)
try await server.start()
And so here I am, posting this. If you think it's a valuable addition, feel free to add it in the library ๐ป
I'd like to use this server in some tests. The tests might be executed in parallel, so I'm not able to hard-code specific port numbers.
If you bind
a socket address with port number 0, the system will assign a unique port number for you, which is ideal for these situations. Unfortunately, AFAICT, there is no way to get the port number or socket from HTTPServer
, so I wouldn't be able to actually make requests to that server instance.
Vapor and Hummingbird both have a great feature where you can specify post parameters as part of the URL/Route using the form:
/ax/objects/:id
Where :id
is accessible as a parameter that is replaced by the passed value.
It would be great to have this same support in FlyingFox at some point!
I noticed the Package.swift does not support watchOS, but the code seems to build without a problem on watchOS, this commit adds it and the result worked fine at least for how I was using it: EmergeTools@30a96b4
I didn't test all features, just the ones I'm using for SnapshotPreviews here: https://github.com/EmergeTools/SnapshotPreviews-iOS/blob/main/Sources/SnapshottingSwift/Snapshots.swift
Is support for this something the maintainers would be interested in adding?
Hi! First of all, You've done an amazing job with this library. Thank you very much.
I was wondering if there's a cleaner way to see if HTTPServer
is active or not. Current work around seems to be either using a custom wrapper around HTTPServer
or use self.listeningAddress != nil
.
Let me know what you think
Althought we can use nginx/apache with a SSL certificate I am wondering if there are plans to add it here in the future.
First, thank you for developing this fantastic lightweight server!
We've encountered a recurring crash, primarily when the iOS app is running in the background. The crash log is as follows:
Crashed: com.apple.root.user-initiated-qos.cooperative
0 libsystem_kernel.dylib 0x1e64 close + 8
1 App 0x71c300 Socket.close() + 4384424704
2 App 0x6f7eb8 HTTPServer.start() + 4384276152
3 libswift_Concurrency.dylib 0x41948 swift::runJobInEstablishedExecutorContext(swift::Job*) + 416
4 libswift_Concurrency.dylib 0x42868 swift_job_runImpl(swift::Job*, swift::ExecutorRef) + 72
5 libdispatch.dylib 0x15944 _dispatch_root_queue_drain + 396
6 libdispatch.dylib 0x16158 _dispatch_worker_thread2 + 164
7 libsystem_pthread.dylib 0xda0 _pthread_wqthread + 228
8 libsystem_pthread.dylib 0xb7c start_wqthread + 8
To provide further context, here are relevant code snippets related to how we start and stop the server:
func start() {
Task {
do {
if await !server.isListening {
try await server.start()
}
} catch (_ as CancellationError) {
print("Task cancelled")
} catch {
logger.error("Error local server: \(error.localizedDescription)")
}
}
}
func stop() {
let task = Task {
await server.stop()
}
task.cancel()
}
Thanks!
JSON responses are common for a web server. With this extension (serializing dates as millis by default to ease JS clients):
extension HTTPResponse {
static let jsonTimestampMillisEncoder: JSONEncoder = {
let jsonEncoder = JSONEncoder()
jsonEncoder.dateEncodingStrategy = .millisecondsSince1970
return jsonEncoder
}()
static func json(_ encodable: Encodable, jsonEncoder: JSONEncoder = jsonTimestampMillisEncoder) throws -> HTTPResponse {
HTTPResponse(statusCode: .ok, body: try jsonEncoder.encode(encodable))
}
}
one can easily return a JSON ready to be consumed by another application:
struct Cat: Codable {
let birthDate: Date
let name: String
}
struct MyHandler : HTTPHandler {
func handleRequest(_ request: HTTPRequest) async throws -> HTTPResponse {
try .json(Cat(birthDate: Date(), name: "Loris"))
}
}
If you think it's a valuable addition, feel free to add it in the library ๐ป
A path traversal vulnerability occurs when paths aren't properly normalised by the server, allowing attackers to access any file on the server, not just files within the served directory.
The following code snippet for creating a simple file server is vulnerable to path traversal (and so is any other code that uses DirectoryHTTPHandler
).
let directoryHandler = DirectoryHTTPHandler(root: URL(fileURLWithPath: "."), serverPath: "/")
let server = HTTPServer(port: 80)
await server.appendRoute("GET *", to: directoryHandler)
try await server.start()
To observe the effect of this vulnerability, run the code snippet above and then run the following command in terminal to see that the server has exposed the contents of your machine's /etc/passwd
file (and all other files for that matter):
curl --path-as-is http://127.0.0.1/../../../../../../../../../../../etc/passwd
Any user that can access the web server can run a similar command on their machine to access any file on the server (within the limits of the privileges that the server is running with).
Abort any requests to the directory handler that contain ../
in their path. This is how vapor's file server middleware avoids path traversal (the following snippet is taken from vapor source). This only fixes the issue for the built-in directory handler, but the check could be applied to all requests (not just those handled by the directory handler) to fix that.
// protect against relative paths
guard !path.contains("../") else {
return request.eventLoop.makeFailedFuture(Abort(.forbidden))
}
I don't like this solution because it feels like a bit of a quick fix, but another way of looking is that its simplicity makes it very hard to mess up.
Normalize request paths in HTTPDecoder
to remove any ..
components. Refer to the RFC specification on removing dot components (https://www.rfc-editor.org/rfc/rfc3986#section-5.2.4).
This solution is a bit more involved, but it results in nicer behaviour in my opinion, and I think it's the approach that most http servers take.
Let me know which solution you prefer. If you would like me to implement the fix, I'd be happy to implement either solution. Personally I prefer the behaviour of the second solution but the simplicity of the first, so it's up to you.
I'm working on a solution for #89, and I've noticed that this property is typed as a Component. I'd love to know more before I start making changes?
We currently only support for
public mutating func appendRoute(_ route: HTTPRoute, to handler: HTTPHandler) {
handlers.append((route, handler))
}
public mutating func appendRoute(_ route: HTTPRoute,
handler: @Sendable @escaping (HTTPRequest) async throws -> HTTPResponse) {
handlers.append((route, ClosureHTTPHandler(handler)))
}
Can we also have the function that inserts at 0 or remove all handlers.
The scenario we encounter is that we use it on the UI test, while we need to mock a response change, we will need to change the handlers array to make it happen, if append is the only function, we will need to rebuild a server again to make the change.
I have a use case where I decrypt large (4-8GB) video files on device and want to allow the decrypted media to be written to the client chunk by chunk so that the app does not exhaust available memory. Is that possible with this library to say have an http handler tied to a route that lets you write to the client chunk by chunk and then close the connection when finished? I see the FlyingSocks looks like it might allow that but Im not sure how to use that with a HTTPHandler tied to the web server.
Im hoping it would be possible because the library I'm using presently does not allow streaming to the http client piece by piece unfortunately.
When it comes to cross-origin requests, I've noticed that the requests don't even reach the server, so I can't set the necessary headers for cross-origin as I usually would. How can I handle this?
private func startListener() async{
do {
let headers = [
HTTPHeader("Access-Control-Allow-Origin"):"*",
HTTPHeader("Access-Control-Allow-Headers"): "*",
HTTPHeader("Access-Control-Allow-Methods"): "OPTIONS, GET, POST",
HTTPHeader("Access-Control-Allow-Credentials") : "true"
]
await server.appendRoute("/hook") { request in
// Don't receive any message.
return HTTPResponse(statusCode: .ok,headers: headers)
}
try await server.start()
print("Server is running on port xxxx")
} catch {
print("Server start error: \(error)")
}
}
I'm using this in a UI test environment. It seems like there's a silent issue at times after starting a server for UI testing where continued execution ceases at the log "starting server port". I have tried:
Starting a server once in a task as documented in readme and stopping it once with the test bundle for the entire run. Issues launching the test app persist using this method.
Starting a server directly using the async setup methods for a given test case. In this case it hangs up on execution.
Starting a server in a task alongside launching the app in a setup method. This method works when I launch the app first, although not always.
Is anyone else using FF as a dependency of UI testing and have found a happiest path for using in such an environment?
If the data passed to Socket.write
is a slice with a non-zero startIndex
, memory after the end of the data buffer will be leaked to the recipient.
The issue is on line 229 of Socket.swift:
let sent = try write(buffer.baseAddress! + index, length: data.endIndex - index)
The code assumes that buffer.baseAddress! + index
correctly gets the byte at index
in the data, however baseAddress
points to the byte at startIndex
not at index 0. For example, consider the following code:
let data = "abcd".data(using: .utf8)!
let slice = data[2...] // contains "cd"
let index = slice.startIndex
try slice.withUnsafeBytes { buffer in
_ = try write(buffer.baseAddress! + index, length: data.endIndex - index)
// (1) (2) (3)
//
// 1. baseAddress points to "c"
// 2. after adding startIndex (which is 2), the pointer points to the byte after the end of the buffer
// 3. length is 2 (4 - 2)
//
// In this example scenario, the server accidentally sends two bytes of
// the memory after the end of the data buffer to the client, which could
// lead to sensitive data being leaked in certain setups. It could also potentially
// be combined with certain other types of vulnerabilities to execute arbitrary code.
}
First, run the following command to start a tcp listener that simply prints out any data it receives.
nc -l 8080
Next, run this code snippet with swift run
for the highest chance of reproducing, because that's how I ran it:
@main
struct FlyingFoxDataTest {
static func main() async throws {
// Generate some dummy data and make a slice
let data = String(repeating: "abcd", count: 32).data(using: .utf8)!
let slice = data[64...]
// This length of string seems to work consistently on my laptop
let secretPassword = "thisismyverylongandsecuresecretpasswordthisismyverylongandsecuresecretpasswordthisismyverylongandsecuresecretpassworditissogood!".data(using: .utf8)!
// Attempt to send the slice through the socket
let socket = try await AsyncSocket.connected(to: .inet(ip4: "127.0.0.1", port: 8080), pool: .polling)
try await socket.write(slice)
}
}
When I run that snippet, I get the following output:
$ nc -l 8080
thisismyverylongandsecuresecretpasswordthisismyverylongandsecure
As you can see, it sent the first half of secretPassword
instead of the contents of the data slice. This bug could have pretty bad side-effects if it appeared in any unfortunate situations.
Make the following change:
- let sent = try write(buffer.baseAddress! + index, length: data.endIndex - index)
+ let sent = try write(buffer.baseAddress! + index - data.startIndex, length: data.endIndex - index)
I'll make a PR to fix this soon.
And I know this full on bug report might be a bit overkill, but I had fun getting that proof of concept to work, so I did it anyway.
Hi Simon,
I'm running into an issue with FlyingFox when building my project. The problem only arises when I specifically direct Swift to build it for macOS 12.0 (the reason I'm doing that is because I'm installing my tool through mint, which automatically targets the machine's os version). Here's the error:
$ swift build -c release -Xswiftc -target -Xswiftc x86_64-apple-macosx12.0
Building for production...
/Users/stackotter/Desktop/Projects/Swift/Scute/.build/checkouts/FlyingFox/Sources/Handlers/ProxyHTTPHandler.swift:51:42: error: ambiguous use of 'data'
let (data, response) = try await session.data(for: req)
^
/Users/stackotter/Desktop/Projects/Swift/Scute/.build/checkouts/FlyingFox/Sources/URLSession+Async.swift:42:10: note: found this candidate
func data(for request: URLRequest, forceFallback: Bool = false) async throws -> (Data, URLResponse) {
^
Foundation.URLSession:3:17: note: found this candidate
public func data(for request: URLRequest, delegate: URLSessionTaskDelegate? = nil) async throws -> (Data, URLResponse)
^
[3/7] Compiling FlyingFox AsyncSequence+Extensions.swift
It seems like you may have to rename your method or change the function signature in some other way so that it doesn't clash with the macOS 12.0 API additions,
Cheers,
stackotter
Is FlyingFox able to handle large file uploads?
When multiple endpoints are defined, one ends up with:
await server.appendRoute(HTTPRoute(.GET, "/cat"), to: MyCatGetHandler())
await server.appendRoute(HTTPRoute(.GET, "/dog"), to: MyDogGetHandler())
await server.appendRoute(HTTPRoute(.GET, "/fish"), to: MyFishGetHandler())
and then implementations contains only the handler:
struct MyCatGetHandler : HTTPHandler {
func handleRequest(_ request: FlyingFox.HTTPRequest) async throws -> FlyingFox.HTTPResponse { }
}
struct MyDogGetHandler : HTTPHandler {
func handleRequest(_ request: FlyingFox.HTTPRequest) async throws -> FlyingFox.HTTPResponse { }
}
struct MyFishGetHandler : HTTPHandler {
func handleRequest(_ request: FlyingFox.HTTPRequest) async throws -> FlyingFox.HTTPResponse { }
}
Maybe it's my personal taste, but I find it better to have both the route and the handler in the same file, so by adding those extensions:
protocol RouteHandler : HTTPHandler {
func route() -> HTTPRoute
}
extension HTTPServer {
func appendRoutes(_ routeHandlers: RouteHandler...) {
routeHandlers.forEach {
appendRoute($0.route(), to: $0)
}
}
}
I'm able to write each handler like this (which I find convenient, specially when the route has placeholders, so I can write a single common function and define shared constants between the route and the request handler):
struct MyCatGetHandler : RouteHandler {
func route() -> HTTPRoute {
HTTPRoute(method: .GET, path: "/cat")
}
func handleRequest(_ request: FlyingFox.HTTPRequest) async throws -> FlyingFox.HTTPResponse {
// implementation
}
}
and also be able to add them all like this:
await server.appendRoutes(
MyCatGetHandler(),
MyDogGetHandler(),
MyFishGetHandler()
)
If you think it's a valuable addition, feel free to add it in the library ๐ป
This is my third proposal so far (#50 #49), and the fact I was able to write everything I needed as an extension denotes really good design choices on your side, so kudos for the great work! ๐ฏ
Thanks for the great software. A small leak found with leaks
:
public static func makeAddress(from addr: sockaddr_storage) throws -> Address {
switch Int32(addr.ss_family) {
case AF_INET:
var addr_in = try sockaddr_in.make(from: addr)
let maxLength = socklen_t(INET_ADDRSTRLEN)
let buffer = UnsafeMutablePointer<CChar>.allocate(capacity: Int(maxLength))
try Socket.inet_ntop(AF_INET, &addr_in.sin_addr, buffer, maxLength)
return .ip4(String(cString: buffer), port: UInt16(addr_in.sin_port).byteSwapped)
case AF_INET6:
var addr_in6 = try sockaddr_in6.make(from: addr)
let maxLength = socklen_t(INET6_ADDRSTRLEN)
let buffer = UnsafeMutablePointer<CChar>.allocate(capacity: Int(maxLength))
try Socket.inet_ntop(AF_INET6, &addr_in6.sin6_addr, buffer, maxLength)
return .ip6(String(cString: buffer), port: UInt16(addr_in6.sin6_port).byteSwapped)
...
}
buffer
is leaked. Should be an easy fix.
Can we setup a local HTTPS Server in Mac OS with FlyingFox?
Hello,
for one of my usecases i need url which uses percent encoded spaces, eg: http://host/some%20folder,
currently such url wont get matched and handled, far from ideal but i got around it for now by editing HTTPDecoder and replacing the "%20" with "+", then i get the route matched and handle it accordingly.
let comps = status
.trimmingCharacters(in: .whitespacesAndNewlines)
.replacingOccurrences(of: "%20", with: "+")
.split(separator: " ", maxSplits: 2, omittingEmptySubsequences: false)
any ideas on how to solve it nicely?
thanks for the great work!
First of all thanks for developing this amazing package!
I wanted to ask if it is possible to use http 2 with this package as I see the server is currently using HTTP/1.1.
If I try to use FlyingFox and embed some HTML with CSS in my app and then load the content via a WebKit browser instance, the HTML content does not load correctly because the CSS does not get loaded.
It appears that the makeContentType
method in FileHTTPHandler
does not return the right type for CSS files. If you add the following case to the method, the HTML content renders correctly:
case "css":
return "text/css"
I was migrating an HTTP Server to FlyingFox and I encountered a client side error "Socket write failed". I spent some time to discover this line:
The client app didn't send the "keepalive" header in requests. I think it would be great to mention "keepalive" in readme.
Thank you for this great lightweight server :)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.