Quantile Regression unable to regress when quantile < 0.5 about caviar HOT 1 CLOSED

Possible solution: (but no time for that)

It seems like you want to use gradient descent on a loss function that involves an indicator function, where the indicator function is defined as:

I(constant <= x)

The indicator function takes the value 1 if the condition (constant <= x) is true, and 0 otherwise. However, the indicator function is not differentiable, as it has a discontinuity at the point where the condition changes from false to true. Gradient descent relies on computing the gradient of the loss function, which requires the function to be differentiable.

You can handle this situation by approximating the indicator function with a smooth, differentiable function. One common approach is to use the sigmoid function:

sigmoid(x) = 1 / (1 + exp(-x))

To construct an approximation of the indicator function, you can scale and shift the input to the sigmoid function:

I_approx(x) = sigmoid(k * (x - constant))

In this expression, k is a positive constant that determines how closely the approximation resembles the indicator function. As k increases, the approximation becomes more similar to the indicator function. However, the gradient of the sigmoid function becomes smaller as k increases, which can make gradient descent slower and more challenging.

Here's a full example:

Define the loss function L(y, y_hat), where y is the true label and y_hat is the predicted value.
Replace the indicator function I(constant <= x) with the approximation I_approx(x) = sigmoid(k * (x - constant)).
Compute the gradient of the loss function with respect to the model parameters.
Update the model parameters using gradient descent.
Keep in mind that the choice of k will affect the trade-off between the accuracy of the approximation and the optimization process. Experiment with different values of k to find a suitable balance for your problem.

from caviar.