workspace and pivot shouldn't be scalar buffers; they should be arrays instead. You want to do something like the following:
func invert(matrix : [Double]) -> [Double] {
var inMatrix = matrix
var N = __CLPK_integer(sqrt(Double(matrix.count)))
var pivots = [__CLPK_integer](count: Int(N), repeatedValue: 0)
var workspace = [Double](count: Int(N), repeatedValue: 0.0)
var error : __CLPK_integer = 0
dgetrf_(&N, &N, &inMatrix, &N, &pivots, &error)
dgetri_(&N, &inMatrix, &N, &pivots, &workspace, &N, &error)
return inMatrix
}
The fact that the current version works for 2x2 matrices is basically dumb luck.