Since the package is not installable as registered packages for now, either register it (Registrator would be the best option) or add the installation steps at the beginning of the README

When __precompile__(false) is removed, RAFF.jl failed in some very simple automated tests. Maybe this has something with this comment about Modules.

In order to add it to production, this parameter has to be eliminated.

The following DOI is missing:

https://link.springer.com/article/10.1007%2Fs11750-010-0169-1

Related to openjournals/joss-reviews#1385

This issue is raised as part of the review process for openjournals/joss-reviews#1385

The Project.toml file still has 0.4.0 as version number, it should be changed to 0.4.1 which corresponds to the latest release

The voting strategy in praff does not apply the filter strategy, nor uses the Inf norm. My suggestion is to implement a function voting_strategy() or something similar so both functions call this one and the implementations remain the same.

We have to decide how RAFF will show output in a mathematical and efficient way, trying to avoid the use of print or @printf as much as possible. Also we have to decide how logging will work in distributed environments.

Currently, those functions are defined as scripts. It is time to move them to RAFF and add tests.

There is an error of acess in some index of array.

The model in this case was model(x,t)=x[1]*t^2+x[2] and the dataset was

A=[ -2.0  5.0 
 -1.5  3.25
 -1.0  2.0 
 -0.5  1.25
  0.0  1.0 
  0.5  1.25
  1.0  2.0 
  1.5  3.25
  2.0  5.0 ]

It would be more interesting if logging messages were omitted by default. In current version info messages are printed. Of course we can set
global_logger(ConsoleLogger(stdout, Logging.Error))
But that's uncomfortable.

I have finished the first draft of the paper. @evcastelani Can you build it using Pandoc to see if it works? We can discuss about improvements in this issue.

When killing praffwith CTRL+C, it got stuck in this

┌ Error: Error when retrieving solutions.
│   e = InterruptException:
└ @ RAFF ~/personal/clss/RAFF.jl/src/RAFF.jl:760

error.

As algorithm lmlovo walks through convergence, we observed that the order of $F_i$'s is maintained. Implement this feature in future releases.

This repo has not seen action in some time. Should it be used?

Add the number of function evaluations and the number of Jacobian evaluations in RAFFOutput object.

RAFF.SortFun!
From Julia style guide, use CamelCase style only for modules and types, use snake_case with lower letters and underscores for functions, values, etc

As recommended in #18 , change matrix data by two arguments x and y.

Run with Julia opened in the examples/ folder:

julia> include("./runraff.jl")
[ Info: Precompiling RAFF [4aa82a78-ed18-41f9-aee6-9d73ba3a0b42]
ERROR: LoadError: MethodError: no method matching raff(::getfield(Main, Symbol("##3#4")), ::Array{Any,2}, ::Int64)
Closest candidates are:
  raff(::Function, ::Array{Float64,2}, ::Int64; kwargs...) at /home/mbesancon/.julia/dev/RAFF/src/RAFF.jl:516
  raff(::Function, ::Function, ::Array{Float64,2}, ::Int64; MAXMS, SEEDMS, initguess, ε, noutliers) at /home/mbesancon/.julia/dev/RAFF/src/RAFF.jl:386
Stacktrace:
 [1] top-level scope at /home/mbesancon/.julia/dev/RAFF/examples/runraff.jl:34
 [2] include at ./boot.jl:326 [inlined]
 [3] include_relative(::Module, ::String) at ./loading.jl:1038
 [4] include(::Module, ::String) at ./sysimg.jl:29
 [5] include(::String) at ./client.jl:403
 [6] top-level scope at none:0
in expression starting at /home/mbesancon/.julia/dev/RAFF/examples/runraff.jl:9

There should be some explanation in the README what the examples and scripts folders are for, are they different? Should they be tested by Travis CI to ensure they stay up to date?

This issue is raised as part of openjournals/joss-reviews#1385

The structure of the JOSS branch is very different from the normal repo. The joss branch should have the same structure as the master branch (that of a Julia package), with a paper folder containing the paper.md and .bib files.

This issue is raised as part of: openjournals/joss-reviews#1385

The algorithm could be another choice, instead of lmlovo. This will require RAFF to accept the inner solver as a choice.

Follow the ideas of Moré, 1978, and implement the smart way of using QR factorization to solve the linear systems.

You will find below my initial review of the paper itself:

Title of the package:
Why "Algebraic"? It uses numerical methods and not symbolic differentiation.

First line: use \mathbb{R} instead of \mathcal{R}

Prefer Package.jl to Package to designate it

Second paragraph of "Summary"
"adjustment of mathematical functions to data" is also refered to as regression, which is not mentioned in the paper, while it is fundamentally the purpose of the package.

"Detection of outliers is always regarded ...". It's hard to see what the authors mean here. Without proper references backing up the statement, this seems rather vague.

"Recently, a good review", the use of "good" is subjective (what is a good review, in what context). A review of what?

In statistics, it is common to use f(x; θ) for a function of x with parameters theta, which one tries to fit in a regression task.

In paragraph "Functionality", the equation is not homogeneous with the rest:
ϕ(x,t) instead of ϕ(x;t), prefer ";" to indicate the separation of data and parameters.

There could be a section on the method itself, or at least more development, including on the LOVO algorithm. For instance, the method is not black-box optimization but assumes the existence of gradient, thus making it first-order. Furthermore, if gradients are not provided by users, the use of ForwardDiff makes it mandatory to have the function defined in a generic way (accepting any Real number).
Thus, how is the proposed method different from using other first-order methods (such as the ones in Optim.jl or JuliaSmoothOptimizers)? This should be detailed.

"it is obvious that its arguments [...]", avoid the use of "obvious" which is subjective, and develop what is meant here.

"Additional features" paragraph, sentence: "the distributed version is a centralized ..." is a contradiction and should be re-phrased.

This belongs to the JOSS review openjournals/joss-reviews#1385

Function LovoFun is a closure with an allocated vector ind which returns the p smallest indices in the LOVO sense. However, if LovoFun is called for two different values of \theta, it will override the indices from the previous calls.

Although this effect is not wrong, it is hidden from the user. We need to make it explicit. This enhancement is necessary to deal with #3 .

Implement the sufficient decrease test, as described in the theoretical description of the algorithm.

Implement some tests from

R. Andreani, G. Cesar, R. M. Cesar-Jr., J. M. Martínez, and P. J. S. Silva, “Efficient curve detection using a {Gauss-Newton} method with applications in agriculture,” in Proc. 1st International Workshop on Computer Vision Applications for Developing Regions in Conjunction with ICCV 2007-CVDR-ICCV07, 2007.

When a worker is created, a MersenneTwister object is assigned to this worker, which is used to generate random starting guesses to lmlovo. However, this object is not reset between two runs with different values of p. That means that the order that each worker gets a problem affects the starting point and, as a consequence, the solution. Therefore, two calls of praff might return different solutions.

This is not a problem in the parallel world, but can be fixed if we restart the MersenneTwister object before the solution of each value of p. In order to do this, we have to send SEEDMS integer, rather than the seedMS object to the worker. Then, we create the MersenneTwister before a solution for a given p.

I think that the documentation could benefit from more information in the overview. Something like the following part of the second paragraph from the Summary section of the paper:

RAFF provides automatic detection of outliers using a voting system. It is an optimization-based package, based on algorithms for Lower Order-Value Optimization (LOVO) which were introduced in [@Andreani2005] and revisited in [@Andreani2009] to fit the user-provided models $\phi$ to experimental data. Recently, a good review can be found in [@Martinez2012]. In order to find a robust adjustment, a voting system is used, which is also responsible for the detection of possible outliers.

Related to openjournals/joss-reviews#1385

From the documentation tutorial:

julia> gmodel(x, t, g)=begin
          g[1] = t[1]^2
          g[2] = 1.0
          return g
       end

Following the Julia style guide, the function could have a bang gmodel! to indicate it is
modifying its arguments. Moreover, the mutated argument is preferred placed first, as in gmodel!(g, x, t)

Issue created for openjournals/joss-reviews#1385

The authors should add some general guidelines for people who want to contribute by either submitting PRs or issues. See https://joss.readthedocs.io/en/latest/submitting.html#submission-requirements
How do people reach out? On the repo or on another platform?

This issue is raised as part of openjournals/joss-reviews#1385