jlouis / fuse Goto Github PK

A Circuit Breaker for Erlang

License: MIT License

Erlang 98.18% Makefile 0.16% CUE 1.51% Nix 0.15%

fuse's Issues

Move from Travis CI to GitHub Actions?

Would you be interested in a pull request that moves CI from Travis CI to GitHub Actions, or at least adds GitHub Actions alongside Travis CI? It seems travis-ci.org will stop being useful from Dec 31 2020. I can do it, except that I think you need to create the workflow yml, first, in master, after which I can branch off of.

gulp build issue with html files

Hi,
I'm having troubles with gulp build.
I'va added a src/app/mystuff directory with a bunch of directives in it like

src/app/mystuff/directive1/directive1.controller.js
src/app/mystuff/directive1/directive1.hml
src/app/mystuff/directive1/directive1.module.js
src/app/mystuff/directive1/i18n/en.json

and so on

gulp build runs fine and outputs no errors but when I access the "compiled" website the browser tells me that directive1.html can't be found.
js files, scss files and json files are correctly copied/injected

As a workaraound I've manually copied the html file in dist/app/mystuf/directive1/

Am I missing something basic ?

thanks a lot !

giuliano

Project name isn't great

I'll open this issue because it's too oblivious (:

Reading announce on erlang-questions@ I wondering how all of those text is related to FUSE and actually, the first and only association was: "Isn't this Erlang bindings to FUSE?" Like fuserl is. Google agrees with me on that: any "{language_name} fuse" request returns list of FUSE bindings on the top, so it would be hard to find your library (though it's looks very useful) like his name will confuse people who doesn't know about.

So may be it's a little bit late for name changes when 1.0.0 is released, but still is it possible to pick some more better name for the project?

Reinstall failures.

Shrinking xxxxxxxxx.xxxx...x.x.x.x...x.xxxxxxxx...xxxxxxxxxxxxxx....xxxxxxxxxxxxxxxxxxxxx(18 times)
[{set,
     {var,1},
     {call,fuse_eqc,install,
         [vanessa,{{standard,1,1},{reset,1}}],
         [{id,3},
          {self,{var,{pid,root}}},
          {res,ok},
          {callouts,
              {seq,
                  [{seq,
                       [{callout,fuse_time,convert_time_unit,
                            [1,milli_seconds,native],
                            1},
                        {return,1}]},
                   {seq,
                       [{internal,fuse_eqc,install_fuse,
                            [vanessa,
                             {vanessa,
                                 {fuse,standard,1,1,ok,undefined,
                                     [{delay,1}],
                                     false}}],
                            [{id,3},{self,{var,{pid,root}}}],
                            empty},
                        {seq,
                            [{internal,fuse_eqc,clear_blown,
                                 [vanessa],
                                 [{id,3},{self,{var,{pid,root}}}],
                                 empty},
                             {seq,
                                 [{internal,fuse_eqc,clear_melts,
                                      [vanessa],
                                      [{id,3},{self,{var,{pid,root}}}],
                                      empty},
                                  {return,ok}]}]}]}]}}]}},
 {set,
     {var,2},
     {call,fuse_eqc,circuit_disable,
         [vanessa],
         [{id,6},
          {self,{var,{pid,root}}},
          {res,ok},
          {callouts,
              {seq,
                  [{internal,fuse_eqc,add_disabled,
                       [vanessa],
                       [{id,6},{self,{var,{pid,root}}}],
                       empty},
                   {seq,
                       [{internal,fuse_eqc,clear_melts,
                            [vanessa],
                            [{id,6},{self,{var,{pid,root}}}],
                            empty},
                        {seq,
                            [{internal,fuse_eqc,clear_blown,
                                 [vanessa],
                                 [{id,6},{self,{var,{pid,root}}}],
                                 empty},
                             {return,ok}]}]}]}}]}},
 {set,
     {var,3},
     {call,fuse_eqc,install,
         [vanessa,{{fault_injection,1.0e-19,1,1},{reset,1}}],
         [{id,21},
          {self,{var,{pid,root}}},
          {res,ok},
          {callouts,
              {seq,
                  [{seq,
                       [{callout,fuse_time,convert_time_unit,
                            [1,milli_seconds,native],
                            1},
                        {return,1}]},
                   {seq,
                       [{internal,fuse_eqc,install_fuse,
                            [vanessa,
                             {vanessa,
                                 {fuse,
                                     {fault_injection,1.0e-19},
                                     1,1,ok,undefined,
                                     [{delay,1}],
                                     false}}],
                            [{id,21},{self,{var,{pid,root}}}],
                            empty},
                        {seq,
                            [{internal,fuse_eqc,clear_blown,
                                 [vanessa],
                                 [{id,21},{self,{var,{pid,root}}}],
                                 empty},
                             {seq,
                                 [{internal,fuse_eqc,clear_melts,
                                      [vanessa],
                                      [{id,21},{self,{var,{pid,root}}}],
                                      empty},
                                  {return,ok}]}]}]}]}}]}},
 {set,
     {var,4},
     {call,fuse_eqc,ask_installed,
         [vanessa],
         [{id,28},
          {self,{var,{pid,root}}},
          {res,ok},
          {callouts,
              {seq,
                  {res,
                      {seq,
                          {res,
                              {val,0.6575722234945981,
                                  {callout,fuse_rand,uniform,[],
                                      0.6575722234945981}},
                              0.6575722234945981},
                          {return,ok}},
                      ok},
                  {return,ok}}}]}}]

S: [{fuse_time_eqc, {state, 0, [], 0}},
    {fuse_eqc, {state, [], []}}]


fuse_eqc:install(vanessa, {{standard, 1, 1}, {reset, 1}}) ->
  1 = fuse_time:convert_time_unit(1, milli_seconds, native),
  ok.

S: [{fuse_time_eqc, {state, 0, [], 0}},
    {fuse_eqc,
       {state, [],
          [{vanessa,
              {fuse, standard, 1, 1, ok, undefined, [{delay, 1}], false}}]}}]


fuse_eqc:circuit_disable(vanessa) -> ok.

S: [{fuse_time_eqc, {state, 0, [], 0}},
    {fuse_eqc,
       {state, [],
          [{vanessa,
              {fuse, standard, 1, 1, ok, undefined, [{delay, 1}], true}}]}}]


fuse_eqc:install(vanessa, {{fault_injection, 1.0e-19, 1, 1}, {reset, 1}}) ->
  1 = fuse_time:convert_time_unit(1, milli_seconds, native),
  ok.

S: [{fuse_time_eqc, {state, 0, [], 0}},
    {fuse_eqc,
       {state, [],
          [{vanessa,
              {fuse, {fault_injection, 1.0e-19}, 1, 1, ok, undefined,
                 [{delay, 1}], false}}]}}]


fuse_eqc:ask_installed(vanessa) -> blown.

S: [{fuse_time_eqc, {state, 0, [], 0}},
    {fuse_eqc,
       {state, [],
          [{vanessa,
              {fuse, {fault_injection, 1.0e-19}, 1, 1, ok, undefined,
                 [{delay, 1}], false}}]}}]


Reason:
  Post-condition failed:
  Failed postcondition: common: blown /= ok
  Callout mismatch: expected: fuse_rand:uniform()

Fault Injection Fuses lead to errors

Hi!
I was just playing with the library and it seems like fault injection fuses lead to an internal error with the fuse monitor (Elixir Crashlog, nevermind the syntax:

(fuse) src/fuse_monitor.erl:91: :fuse_monitor.update({Catalog.HTTPGateway, {:gradual, 0.002}}, [])

Looking at fuse_monitor.erl the error looks like a missing implementation of update for fault injection fuses.

Model failure: when resetting a fuse the timer reference is not reset

When executing a fuse_reset command, the timer is not changed to the state undefined. This is a mistake which makes the model break an assertion: "At any point in time, there is at most one timer set on a fuse". Correcting this is necessary for ongoing correctness checking of fuses.

S: [{fuse_time_eqc,
       {state, 1000, [{1, 0, '_', {reset, phineas}}], 1}},
    {fuse_eqc,
       {state, [{phineas, 0}, {phineas, 0}],
          [{phineas,
              {fuse, {fault_injection, 1.0e-19}, 1, 1, {blown, []}, {tref, 0},
                 [{delay, 1}], false}}]}}]


fuse_eqc:fuse_reset(phineas, {tref, 0}) ->
  false = fuse_time:cancel_timer({tref, 0}),
  ok.

S: [{fuse_time_eqc, {state, 1000, [], 1}},
    {fuse_eqc,
       {state, [],
          [{phineas,
              {fuse, {fault_injection, 1.0e-19}, 1, 1, ok, {tref, 0},
                 [{delay, 1}], false}}]}}]

Add release 16 and 17 support

In order to support R16 and R17, we need to update fuse_rand to be backwards compatible. The fuse_time module already has compatibility support in there. The fix is twofold:

You need to call random not rand.
You need to call random:seed/1 correctly and inject this into the system.

Once this is done, the system will support the gradual fuse types without any trouble.

Control the service restart after blown

Hi,

I am using fuse to control the access to a backend service (DAL API). It would be nice that we have some good way of passing from blown to ok gradually. Otherwise, if the backend is under load (502/503), for example, the requests will be back on charge all at the same time, after the "heal" interval and can cause problems again.

Thank you,

(I will be able to implement a solution if you think that may be useful)

Pedro

Model failure: Unexpected cancel timer

Shrinking xxxxxx.xx.x.x.xxx.xx.x..x.x.xxxx...x...xxxxx(16 times)
[{set,
  {var,1},
  {call,fuse_eqc,install,
   [phineas,{{fault_injection,1.0e-19,1,1},{reset,1}}],
   [{id,1},
    {self,{var,{pid,root}}},
    {res,ok},
    {callouts,
     {seq,
      [{seq,
        [{callout,fuse_time,convert_time_unit,[1,milli_seconds,native],1},
         {return,1}]},
       {seq,
        [{internal,fuse_eqc,install_fuse,
          [phineas,
           {phineas,
            {fuse,
             {fault_injection,1.0e-19},
             1,1,ok,undefined,
             [{delay,1}],
             false}}],
          [{id,1},{self,{var,{pid,root}}}],
          empty},
         {seq,
          [{internal,fuse_eqc,clear_blown,
            [phineas],
            [{id,1},{self,{var,{pid,root}}}],
            empty},
           {seq,
            [{internal,fuse_eqc,clear_melts,
              [phineas],
              [{id,1},{self,{var,{pid,root}}}],
              empty},
             {return,ok}]}]}]}]}}]}},
 {set,
  {var,2},
  {call,fuse_eqc,melt_installed,
   [phineas],
   [{id,18},
    {self,{var,{pid,root}}},
    {res,ok},
    {callouts,
     {seq,
      {res,
       {internal,fuse_time_eqc,monotonic_time,[],
        [{id,18},{self,{var,{pid,root}}}],
        {seq,[{callout,fuse_time,monotonic_time,[],0},{return,0}]}},
       0},
      {seq,
       [{seq,
         [{internal,fuse_eqc,record_melt,
           [phineas,0],
           [{id,18},{self,{var,{pid,root}}}],
           empty},
          {seq,
           {res,
            {internal,fuse_eqc,fuse_period,
             [phineas],
             [{id,18},{self,{var,{pid,root}}}],
             empty},
            1},
           {seq,
            [{internal,fuse_eqc,expire_melts,
              [phineas,1,0],
              [{id,18},{self,{var,{pid,root}}}],
              empty},
             empty]}}]},
        {return,ok}]}}}]}},
 {set,
  {var,3},
  {call,fuse_eqc,melt_installed,
   [phineas],
   [{id,19},
    {self,{var,{pid,root}}},
    {res,ok},
    {callouts,
     {seq,
      {res,
       {internal,fuse_time_eqc,monotonic_time,[],
        [{id,19},{self,{var,{pid,root}}}],
        {seq,[{callout,fuse_time,monotonic_time,[],0},{return,0}]}},
       0},
      {seq,
       [{seq,
         [{internal,fuse_eqc,record_melt,
           [phineas,0],
           [{id,19},{self,{var,{pid,root}}}],
           empty},
          {seq,
           {res,
            {internal,fuse_eqc,fuse_period,
             [phineas],
             [{id,19},{self,{var,{pid,root}}}],
             empty},
            1},
           {seq,
            [{internal,fuse_eqc,expire_melts,
              [phineas,1,0],
              [{id,19},{self,{var,{pid,root}}}],
              empty},
             {internal,fuse_eqc,blow_fuse,
              [phineas],
              [{id,19},{self,{var,{pid,root}}}],
              {seq,
               [{internal,fuse_eqc,add_blown,
                 [phineas],
                 [{id,19},{self,{var,{pid,root}}}],
                 empty},
                {seq,
                 {res,
                  {internal,fuse_eqc,next_command,
                   [phineas],
                   [{id,19},{self,{var,{pid,root}}}],
                   {return,{delay,1}}},
                  {delay,1}},
                 {seq,
                  {res,
                   {internal,fuse_time_eqc,send_after,
                    [1,'_',{reset,phineas}],
                    [{id,19},{self,{var,{pid,root}}}],
                    {seq,
                     [{callout,fuse_time,send_after,
                       [1,'_',{reset,phineas}],
                       {tref,0}},
                      {return,{tref,0}}]}},
                   {tref,0}},
                  {seq,
                   [empty,
                    {internal,fuse_eqc,add_timer,
                     [phineas,{tref,0}],
                     [{id,19},{self,{var,{pid,root}}}],
                     empty}]}}}]}}]}}]},
        {return,ok}]}}}]}},
 {set,
  {var,4},
  {call,fuse_eqc,install,
   [phineas,{{fault_injection,1.0e-19,1,1},{reset,1}}],
   [{id,23},
    {self,{var,{pid,root}}},
    {res,ok},
    {callouts,
     {seq,
      [{seq,
        [{callout,fuse_time,convert_time_unit,[1,milli_seconds,native],1},
         {return,1}]},
       {seq,
        [{internal,fuse_eqc,install_fuse,
          [phineas,
           {phineas,
            {fuse,
             {fault_injection,1.0e-19},
             1,1,ok,undefined,
             [{delay,1}],
             false}}],
          [{id,23},{self,{var,{pid,root}}}],
          empty},
         {seq,
          [{internal,fuse_eqc,clear_blown,
            [phineas],
            [{id,23},{self,{var,{pid,root}}}],
            empty},
           {seq,
            [{internal,fuse_eqc,clear_melts,
              [phineas],
              [{id,23},{self,{var,{pid,root}}}],
              empty},
             {return,ok}]}]}]}]}}]}}]

S: [{fuse_time_eqc, {state, 0, [], 0}},
    {fuse_eqc, {state, [], []}}]


fuse_eqc:install(phineas, {{fault_injection, 1.0e-19, 1, 1}, {reset, 1}}) ->
  1 = fuse_time:convert_time_unit(1, milli_seconds, native),
  ok.

S: [{fuse_time_eqc, {state, 0, [], 0}},
    {fuse_eqc,
       {state, [],
          [{phineas,
              {fuse, {fault_injection, 1.0e-19}, 1, 1, ok, undefined,
                 [{delay, 1}], false}}]}}]


fuse_eqc:melt_installed(phineas) ->
  0 = fuse_time:monotonic_time(),
  ok.

S: [{fuse_time_eqc, {state, 0, [], 0}},
    {fuse_eqc,
       {state, [{phineas, 0}],
          [{phineas,
              {fuse, {fault_injection, 1.0e-19}, 1, 1, ok, undefined,
                 [{delay, 1}], false}}]}}]


fuse_eqc:melt_installed(phineas) ->
  0 = fuse_time:monotonic_time(),
  {tref, 0} = fuse_time:send_after(1, <0.10943.2>, {reset, phineas}),
  ok.

S: [{fuse_time_eqc,
       {state, 0, [{1, 0, '_', {reset, phineas}}], 1}},
    {fuse_eqc,
       {state, [{phineas, 0}, {phineas, 0}],
          [{phineas,
              {fuse, {fault_injection, 1.0e-19}, 1, 1, {blown, []}, {tref, 0},
                 [{delay, 1}], false}}]}}]


fuse_eqc:install(phineas, {{fault_injection, 1.0e-19, 1, 1}, {reset, 1}}) ->
  1 = fuse_time:convert_time_unit(1, milli_seconds, native),
  exit({mocking_error, unexpected}) = fuse_time:cancel_timer({tref, 0}),
  exit({{{{mocking_error, {unexpected, fuse_time:cancel_timer({tref, 0})}},
          [{eqc_mocking, f6092900_0,
              [fuse_time, cancel_timer, [{tref, 0}]],
              [{file, "../../eqc-project/src/eqc_mocking.erl"}, {line, 398}]},
           {eqc_mocking, do_action, 3,
              [{file, "../../eqc-project/src/eqc_mocking.erl"}, {line, 394}]},
           {fuse_server, reset_timer, 1,
              [{file, "src/fuse_server.erl"}, {line, 352}]},
           {fuse_server, handle_call, 3,
              [{file, "src/fuse_server.erl"}, {line, 173}]},
           {gen_server, try_handle_call, 4,
              [{file, "gen_server.erl"}, {line, 615}]},
           {gen_server, handle_msg, 5,
              [{file, "gen_server.erl"}, {line, 647}]},
           {proc_lib, init_p_do_apply, 3,
              [{file, "proc_lib.erl"}, {line, 247}]}]},
         {gen_server, call,
            [fuse_server,
             {install,
                {fuse, phineas, 1, 1, 1, [], {gradual, 1.0e-19}, none,
                   true}}]}},
        [{gen_server, call, 2, [{file, "gen_server.erl"}, {line, 204}]},
         {fuse_eqc, install, 2,
            [{file, "test/fuse_eqc.erl"}, {line, 166}]},
         {eqc_cluster_callbacks, wrap_call, 3,
            [{file, "../../eqc-project/src/eqc_cluster_callbacks.erl"},
             {line, 543}]}]}).

S: [{fuse_time_eqc,
       {state, 0, [{1, 0, '_', {reset, phineas}}], 1}},
    {fuse_eqc,
       {state, [],
          [{phineas,
              {fuse, {fault_injection, 1.0e-19}, 1, 1, ok, undefined,
                 [{delay, 1}], false}}]}}]


Reason:
  Post-condition failed:
  Exception:
    {{{{mocking_error, {unexpected, fuse_time:cancel_timer({tref, 0})}},
       [{eqc_mocking, f6092900_0,
           [fuse_time, cancel_timer, [{tref, 0}]],
           [{file, "../../eqc-project/src/eqc_mocking.erl"}, {line, 398}]},
        {eqc_mocking, do_action, 3,
           [{file, "../../eqc-project/src/eqc_mocking.erl"}, {line, 394}]},
        {fuse_server, reset_timer, 1,
           [{file, "src/fuse_server.erl"}, {line, 352}]},
        {fuse_server, handle_call, 3,
           [{file, "src/fuse_server.erl"}, {line, 173}]},
        {gen_server, try_handle_call, 4,
           [{file, "gen_server.erl"}, {line, 615}]},
        {gen_server, handle_msg, 5,
           [{file, "gen_server.erl"}, {line, 647}]},
        {proc_lib, init_p_do_apply, 3,
           [{file, "proc_lib.erl"}, {line, 247}]}]},
      {gen_server, call,
         [fuse_server,
          {install,
             {fuse, phineas, 1, 1, 1, [], {gradual, 1.0e-19}, none, true}}]}},
     [{gen_server, call, 2, [{file, "gen_server.erl"}, {line, 204}]},
      {fuse_eqc, install, 2,
         [{file, "test/fuse_eqc.erl"}, {line, 166}]},
      {eqc_cluster_callbacks, wrap_call, 3,
         [{file, "../../eqc-project/src/eqc_cluster_callbacks.erl"},
          {line, 543}]}]}
  Callout mismatch: unexpected: fuse_time:cancel_timer({tref, 0})
false
20>

Please include a fully working example

Would it be possible to include a complete example(s) as source?

SASL started by default

I'm coming from the Elixir world and when adding Fuse to my project SASL logs began showing up. This is the only Erlang dependency (of which there are many) for which this happens in my project. I'm curious as to why this is enabled by default and whether SASL logging could be specified in the application configuration.

What is a circuit breaker..?

So this is a circuit breaker for Erlang, that's pretty clear. I assume it has something to do with limiting retries, or debouncing requests, or something. It sounds like circuit breaker is probably a standard term in the Erlang community, but I'm new and don't really understand.

What does it do, and what is it usually used for?

Some invariants which is needed to move forward:

Timers are lock-stepped to the model. So there is always a
relationship between a timer and its occurrence in the model.

(add a timer trigger which is module local and does the real thing)

When a timer triggers, it is removed from the model.
ASSERT: The timer which triggers is the one removed from the model.

(add a way to cancel timers correctly)

When we cancel a timer, it is removed from the model.
The SUT will call cancel_timer in this case.
ASSERT: The timer we cancel is the one removed.

telemetry implementation of fuse_stats_plugin?

Is there an implementation of the fuse_stats_plugin behavior that reports metrics to https://github.com/beam-telemetry/telemetry? I did a Google search and a Github search and didn't find anything. I am standardizing on telemetry for my systems and must move off the :fuse_stats_prometheus plugin I was initially using.

If there is not an implementation, are you aware of any technical reasons why this wouldn't be possible? I may be able to contribute a telemetry implementation if it is desired.

Is sasl needed in the application?

Do we really need to start sasl? Was this left in there intentionally?

update readme - users

The current code status is that we have extensive test cases and test frameworks written around the code, but it has not been used in production systems yet. If you use the system in production, I would very much like to hear about it. Especially if you encountered any problems while doing so.

Your project is used by Yokozuna - part of Riak (which is quite a well known project).

Congrats!

tags not synced

The last tag is 2.2.0.

Generalized support for statistics export to monitoring systems.

How would you feel about removing direct folsom integration from fuse?

My system uses exometer for reporting, and unfortunately also riak_core (which is broken wrt. other applications depending on folsom, Cf. basho/riak_core#636).

Given that the fuse application already exports the state of a particular fuse via fuse:ask/2, and also fuse_event, it seems like there's enough exported to allow users to integrate fuse with whatever metrics system at a higher level.

Wrt. exporting counts of foo.melt, foo.ok, foo.blown, perhaps a call like

fuse:stats(Name :: atom()) -> [proplists:property()].

would be more general.

Testing could be done by including calls to fuse:stats/1 in the generated command sequences of fuse_eqc.erl, with a postcondition verifying that a particular fuse's (modeled) history is reflected accurately in the stats export.

What do you think?

jlouis / fuse Goto Github PK

fuse's Issues

(add a timer trigger which is module local and does the real thing)

(add a way to cancel timers correctly)

Recommend Projects

Recommend Topics

Recommend Org