itislab / alpheus Goto Github PK

View Code? Open in Web Editor NEW

6.0 7.0 5.0 832 KB

Alpheus: A tool for organizing and managing computational experiments

License: MIT License

F# 99.66% Batchfile 0.05% C 0.29% Shell 0.01%

machine-learning computational-science dotnet-core nuget dotnet-cli

alpheus's Issues

NullReferenceException in 'angara compute'

I have following exception:

        System.NullReferenceException: Object reference not set to an instance of an object.
   at [email protected](FSharpOption`1 x) in D:\alpheus\AlpheusCore\ComputationGraph.fs:line 113
   at Microsoft.FSharp.Collections.Internal.IEnumerator.map@74.DoMoveNext(b& curr)
   at Microsoft.FSharp.Collections.Internal.IEnumerator.MapEnumerator`1.System-Collections-IEnumerator-MoveNext()
   at Microsoft.FSharp.Collections.Internal.IEnumerator.map2@102.DoMoveNext(c& curr)
   at Microsoft.FSharp.Collections.Internal.IEnumerator.MapEnumerator`1.System-Collections-IEnumerator-MoveNext()
   at System.Collections.Generic.List`1.AddEnumerable(IEnumerable`1 enumerable)
   at Microsoft.FSharp.Collections.SeqModule.ToArray[T](IEnumerable`1 source)
   at [email protected](FSharpOption`1[] _arg4) in D:\alpheus\AlpheusCore\ComputationGraph.fs:line 115
   at Microsoft.FSharp.Control.AsyncPrimitives.CallThenInvokeNoHijackCheck[a,b](AsyncActivation`1 ctxt, FSharpFunc`2 userCode, b result1)
   at Microsoft.FSharp.Control.Trampoline.Execute(FSharpFunc`2 firstAction)
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.FSharp.Control.AsyncResult`1.Commit()
   at Microsoft.FSharp.Control.AsyncPrimitives.RunSynchronouslyInCurrentThread[a](CancellationToken cancellationToken, FSharpAsync`1 computation)
   at Microsoft.FSharp.Control.AsyncPrimitives.RunSynchronously[T](CancellationToken cancellationToken, FSharpAsync`1 computation, FSharpOption`1 timeout)
   at Microsoft.FSharp.Control.FSharpAsync.RunSynchronously[T](FSharpAsync`1 computation, FSharpOption`1 timeout, FSharpOption`1 cancellationToken)
   at ItisLab.Alpheus.ComputationGraph.CommandMethod.Execute(FSharpList`1 inputs, FSharpOption`1 _arg1) in D:\alpheus\AlpheusCore\ComputationGraph.fs:line 56
   at Angara.Execution.Runtime`1.buildEvaluation(m v, FSharpList`1 index, UInt64 time, State`2 state, CancellationTokenSource cts, Boolean doContinue, Unit unitVar0)

when running alpheus compute irisdata.png command in attached project folder.

repro.zip

"alpheus compute artefact" must restore artefact, if it is missing locally

A user expects that after running the following command

alpheus compute something.txt

there will appear something.txt, if it wasn't there, both if its up-to-date version is in a remote storage or it should be recomputed.

Now it appears only if it is not in a remote storage, i.e. is recomputed, otherwise alpheus does nothing (expecting that the user can restore the file).

working directory is not captured for the graph compute nodes

The command is executed not from the working directory specified in alpheus file but from the working directory of current alpheus invokation.

In particular:
ProcessStartInfo.FileName relative path works not from ProcessStartInfo.WorkingDirectory but from the current alpheus invocation working directory

Workaround:
build all your nodes from the project root folder

Alpheus fails on index "shrinkage"

Let's say one we had 0 .. 10 indices. We saved the artefact, so there is the alph file.

After that we deleted index 10, so there are only 0..9 on disk now.

Check how the system performs on that.

Alpheus comand vertex outputs should be marked with readonly filesystem attribute

upon successful execution

Vector inputs are not resortored upon compute

I get

(venv) PS D:\bengaliai-cv19> alpheus compute .\experiment_outputs\dgrechka_2_mobileNetV2_bottleneck\vector.alph
System.ArgumentException: Index doesn't correspond to the rank of the artefact
Parameter name: index
   at ItisLab.Alpheus.DependencyGraph.LinkToArtefact.AnalyzeStatus(FSharpFunc`2 checkStoragePresence, FSharpList`1 index) in C:\repos\alpheus\AlpheusCore\DependencyGraph.fs:line 248
   at [email protected](FSharpList`1 idx) in C:\repos\alpheus\AlpheusCore\API.fs:line 377
   at Microsoft.FSharp.Collections.Internal.IEnumerator.map@75.DoMoveNext(b& curr) in E:\A\_work\130\s\src\fsharp\FSharp.Core\seq.fs:line 78
   at Microsoft.FSharp.Collections.Internal.IEnumerator.MapEnumerator`1.System-Collections-IEnumerator-MoveNext() in E:\A\_work\130\s\src\fsharp\FSharp.Core\seq.fs:line 64
   at System.Collections.Generic.List`1.AddEnumerable(IEnumerable`1 enumerable)
   at Microsoft.FSharp.Collections.SeqModule.ToArray[T](IEnumerable`1 source) in E:\A\_work\130\s\src\fsharp\FSharp.Core\seq.fs:line 823
   at <StartupCode$FSharp-Core>[email protected](AsyncActivation`1 ctxt) in E:\A\_work\130\s\src\fsharp\FSharp.Core\async.fs:line 1178
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.FSharp.Control.AsyncResult`1.Commit() in E:\A\_work\130\s\src\fsharp\FSharp.Core\async.fs:line 349
   at Microsoft.FSharp.Control.AsyncPrimitives.RunSynchronouslyInCurrentThread[a](CancellationToken cancellationToken, FSharpAsync`1 computation) in E:\A\_work\130\s\src\fsharp\FSharp.Core\async.fs:line 870
   at Microsoft.FSharp.Control.AsyncPrimitives.RunSynchronously[T](CancellationToken cancellationToken, FSharpAsync`1 computation, FSharpOption`1 timeout) in E:\A\_work\130\s\src\fsharp\FSharp.Core\async.fs:line 890
   at Microsoft.FSharp.Control.FSharpAsync.RunSynchronously[T](FSharpAsync`1 computation, FSharpOption`1 timeout, FSharpOption`1 cancellationToken) in E:\A\_work\130\s\src\fsharp\FSharp.Core\async.fs:line 1151
   at [email protected](ArtefactVertex _arg4) in C:\repos\alpheus\AlpheusCore\API.fs:line 375
   at ItisLab.Alpheus.Results.ResultBuilder.Bind[a2,a3,a4](FSharpResult`2 m, FSharpFunc`2 f) in C:\repos\alpheus\AlpheusCore\ResultBuilder.fs:line 16
   at [email protected](Unit _arg3) in C:\repos\alpheus\AlpheusCore\API.fs:line 369
   at ItisLab.Alpheus.Results.ResultBuilder.Bind[a2,a3,a4](FSharpResult`2 m, FSharpFunc`2 f) in C:\repos\alpheus\AlpheusCore\ResultBuilder.fs:line 16
   at [email protected](Unit unitVar) in C:\repos\alpheus\AlpheusCore\API.fs:line 365
   at ItisLab.Alpheus.Results.ResultBuilder.Run[m](FSharpFunc`2 f) in C:\repos\alpheus\AlpheusCore\ResultBuilder.fs:line 31
   at [email protected](Graph _arg1) in C:\repos\alpheus\AlpheusCore\API.fs:line 389
   at Microsoft.FSharp.Control.AsyncPrimitives.CallThenInvokeNoHijackCheck[a,b](AsyncActivation`1 ctxt, FSharpFunc`2 userCode, b result1) in E:\A\_work\130\s\src\fsharp\FSharp.Core\async.fs:line 417
   at [email protected](AsyncActivation`1 ctxt) in C:\repos\alpheus\AlpheusCore\API.fs:line 22
   at [email protected](AsyncActivation`1 ctxt) in C:\repos\alpheus\AlpheusCore\API.fs:line 339
   at Microsoft.FSharp.Control.Trampoline.Execute(FSharpFunc`2 firstAction) in E:\A\_work\130\s\src\fsharp\FSharp.Core\async.fs:line 109
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.FSharp.Control.AsyncResult`1.Commit() in E:\A\_work\130\s\src\fsharp\FSharp.Core\async.fs:line 349
   at Microsoft.FSharp.Control.AsyncPrimitives.RunSynchronouslyInCurrentThread[a](CancellationToken cancellationToken, FSharpAsync`1 computation) in E:\A\_work\130\s\src\fsharp\FSharp.Core\async.fs:line 870
   at Microsoft.FSharp.Control.AsyncPrimitives.RunSynchronously[T](CancellationToken cancellationToken, FSharpAsync`1 computation, FSharpOption`1 timeout) in E:\A\_work\130\s\src\fsharp\FSharp.Core\async.fs:line 890
   at Microsoft.FSharp.Control.FSharpAsync.RunSynchronously[T](FSharpAsync`1 computation, FSharpOption`1 timeout, FSharpOption`1 cancellationToken) in E:\A\_work\130\s\src\fsharp\FSharp.Core\async.fs:line 1153
   at ItisLab.Alpheus.API.compute(String experimentRoot, ArtefactId artefactId) in C:\repos\alpheus\AlpheusCore\API.fs:line 338
   at [email protected](Tuple`2 _arg8) in C:\repos\alpheus\Alpheus.CLI\Runner.fs:line 66
   at ItisLab.Alpheus.Results.ResultBuilder.Bind[a2,a3,a4](FSharpResult`2 m, FSharpFunc`2 f) in C:\repos\alpheus\AlpheusCore\ResultBuilder.fs:line 16
   at [email protected](Unit unitVar) in C:\repos\alpheus\Alpheus.CLI\Runner.fs:line 65
   at ItisLab.Alpheus.Results.ResultBuilder.Run[m](FSharpFunc`2 f) in C:\repos\alpheus\AlpheusCore\ResultBuilder.fs:line 31
   at ItisLab.Alpheus.CliRunner.run(String programName, String workingDir, ParseResults`1 parseResults) in C:\repos\alpheus\Alpheus.CLI\Runner.fs:line 62
   at ItisLab.Alpheus.Program.main(String[] argv) in C:\repos\alpheus\Alpheus.CLI\Program.fs:line 42

while I have inputs (also vector) available in storage

Alph file:

{
  "FileFormatVersion": 1,
  "Origin": {
    "Case": "CommandOrigin",
    "Fields": [
      {
        "Inputs": [
          {
            "RelativePath": "..\\..\\code\\dgrechka\\train_mobileNetV2_bottleneck.py",
            "Hash": null
          },
          {
            "RelativePath": "..\\..\\data\\bengaliai-cv19\\",
            "Hash": null
          },
          {
            "RelativePath": "..\\..\\data\\5foldCvSplits\\*.val_ids.csv",
            "Hash": null
          }
        ],
        "Outputs": [
          {
            "RelativePath": "*\\",
            "Hash": null
          }
        ],
        "ResourceGroups": [
          "gpu"
        ],
        "SuccessfulExitCodes": [
          0
        ],
        "OutputIndex": 0,
        "WorkingDirectory": "..\\..",
        "Command": "python $in1 $in2 $in3 $out1",
        "Signature": "F6CAA8BF1432F2612D8F606B1A3636E90D52E58D",
        "OutputsCleanDisabled": false
      }
    ]
  },
  "IsTracked": false
}

Status for vector graphs

alpheus status must work in case when the dependency graph contains vector operations

Exception during the sucessful finish of scatter vertex element

Getting

Internal error occurred (please report it at https://github.com/itislab/alpheus/issues): Failed to compute the artefacts:
        Newtonsoft.Json.JsonWriterException: Token PropertyName in state Property would result in an invalid JSON object. Path 'Origin.Fields[0].Outputs[0].Hash'.
   at Newtonsoft.Json.JsonWriter.AutoComplete(JsonToken tokenBeingWritten)
   at Newtonsoft.Json.JsonTextWriter.WritePropertyName(String name)
   at [email protected](Tuple`2 tupledArg) in C:\repos\alpheus\AlpheusCore\CustomSerializers.fs:line 24
   at Microsoft.FSharp.Collections.SeqModule.Iterate[T](FSharpFunc`2 action, IEnumerable`1 source) in E:\A\_work\130\s\src\fsharp\FSharp.Core\seq.fs:line 495
   at [email protected](Tuple`2 tupledArg, MdMap`2 value) in C:\repos\alpheus\AlpheusCore\CustomSerializers.fs:line 20
   at ItisLab.Alpheus.CustomSerializers.ArtefactVersionConverter.WriteJson(JsonWriter writer, Object value, JsonSerializer serializer) in C:\repos\alpheus\AlpheusCore\CustomSerializers.fs:line 28
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeConvertable(JsonWriter writer, JsonConverter converter, Object value, JsonContract contract, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeList(JsonWriter writer, IEnumerable values, JsonArrayContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.Serialize(JsonWriter jsonWriter, Object value, Type objectType)
   at Newtonsoft.Json.Converters.DiscriminatedUnionConverter.WriteJson(JsonWriter writer, Object value, JsonSerializer serializer)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeConvertable(JsonWriter writer, JsonConverter converter, Object value, JsonContract contract, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.Serialize(JsonWriter jsonWriter, Object value, Type objectType)
   at Newtonsoft.Json.JsonSerializer.SerializeInternal(JsonWriter jsonWriter, Object value, Type objectType)
   at Newtonsoft.Json.JsonConvert.SerializeObjectInternal(Object value, Type type, JsonSerializer jsonSerializer)
   at ItisLab.Alpheus.AlphFiles.save(AlphFile alphfile, String filepath) in C:\repos\alpheus\AlpheusCore\AlphFiles.fs:line 57
   at [email protected](Unit unitVar0) in C:\repos\alpheus\AlpheusCore\DependencyGraph.fs:line 213
   at ItisLab.Alpheus.DependencyGraph.ArtefactVertex.SaveAlphFile() in C:\repos\alpheus\AlpheusCore\DependencyGraph.fs:line 163
   at [email protected](LinkToArtefact out) in C:\repos\alpheus\AlpheusCore\DependencyGraph.fs:line 413
   at Microsoft.FSharp.Collections.SeqModule.Iterate[T](FSharpFunc`2 action, IEnumerable`1 source) in E:\A\_work\130\s\src\fsharp\FSharp.Core\seq.fs:line 495
   at [email protected](Unit _arg5) in C:\repos\alpheus\AlpheusCore\DependencyGraph.fs:line 413
   at Microsoft.FSharp.Control.AsyncPrimitives.CallThenInvokeNoHijackCheck[a,b](AsyncActivation`1 ctxt, FSharpFunc`2 userCode, b result1) in E:\A\_work\130\s\src\fsharp\FSharp.Core\async.fs:line 417
   at [email protected](AsyncActivation`1 ctxt) in E:\A\_work\130\s\src\fsharp\FSharp.Core\async.fs:line 589
   at Microsoft.FSharp.Control.Trampoline.Execute(FSharpFunc`2 firstAction) in E:\A\_work\130\s\src\fsharp\FSharp.Core\async.fs:line 109
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.FSharp.Control.AsyncResult`1.Commit() in E:\A\_work\130\s\src\fsharp\FSharp.Core\async.fs:line 349
   at Microsoft.FSharp.Control.AsyncPrimitives.RunSynchronouslyInCurrentThread[a](CancellationToken cancellationToken, FSharpAsync`1 computation) in E:\A\_work\130\s\src\fsharp\FSharp.Core\async.fs:line 870
   at Microsoft.FSharp.Control.AsyncPrimitives.RunSynchronously[T](CancellationToken cancellationToken, FSharpAsync`1 computation, FSharpOption`1 timeout) in E:\A\_work\130\s\src\fsharp\FSharp.Core\async.fs:line 890
   at Microsoft.FSharp.Control.FSharpAsync.RunSynchronously[T](FSharpAsync`1 computation, FSharpOption`1 timeout, FSharpOption`1 cancellationToken) in E:\A\_work\130\s\src\fsharp\FSharp.Core\async.fs:line 1151
   at ItisLab.Alpheus.ComputationGraph.CommandMethod.Execute(FSharpList`1 inputs, FSharpOption`1 _arg1) in C:\repos\alpheus\AlpheusCore\ComputationGraph.fs:line 74
   at Angara.Execution.Runtime`1.buildEvaluation(m v, FSharpList`1 index, UInt64 time, State`2 state, CancellationTokenSource cts, Boolean doContinue, Unit unitVar0)

Alph file:

{
  "FileFormatVersion": 1,
  "Origin": {
    "Case": "CommandOrigin",
    "Fields": [
      {
        "Inputs": [
          {
            "RelativePath": "..\\..\\code\\dgrechka\\train_NASNetMobile_full_do_0_5.py",
            "Hash": "6D1D1C524753B759525804A80AF7B9DE6ED847CBC94B295DF246298BC6AFEBB2712B054ACE761500DE7B6EC798F33A958DEB2F0A9886B363BFF03EA3D28CB044"
          },
          {
            "RelativePath": "..\\..\\data\\bengaliai-cv19\\",
            "Hash": "D09946D9348B9135927E3C69F22AFB441F5EA040995229E4119F6B3B6DC10537DE1BF1FA6850A5BCFA1C90849BDA14AEC20B69EC410422F558CDA8CBE0A5362F"
          },
          {
            "RelativePath": "..\\..\\data\\5foldCvSplits\\*.val_ids.csv",
            "Hash": {
              "0": "B09EA6865EFC699090D8983F4A2BC36F82249E1E3163A6D5EB8611524F76BF024102DDCBD4A401BEEEDEB71FE1D5DE1752F338840722624B17BF2B4A3FA3228D",
              "1": "541369BD74236A263A5A7525C712B57836D98C183F442D1C96A366C55D7443FFE7543B7F5A6D11EDCFD1202040D441A2C64861AD0C8CB74307D58ECB6138FD95",
              "2": "377CB26C5DA8AF87EEFBB60383573841774D9329FFA41A8620A877B0B426D03992553A18E41A1C6D841DB3BCBF270A7A9F478A36F20034F3B663909822FB3A6B",
              "3": "5A708C1F023A3B8EDB9E31ADC9D95D66DA1428532643F6EC05F895A55982B0EE30FD0EC741E7DB122ED94A559C10D4A55CC2C6F149FFBAA11E34E85E0EA8B6B0",
              "4": "126EC1D9569D45674A0EB7F9CB1A861F8CBFA854CEBA03061EFE070F004C151F8BFDD72980139891A828E1D03B99BFDE36DBAF0C46F149F7106C0DA04AD20F8F"
            }
          },
          {
            "RelativePath": "..\\dgrechka_2_NASNetMobile_bottleneck\\*\\",
            "Hash": {
              "0": "DD0AAF3FBB7E769D26B0B2954D48ADADE1850FBEB2368327F4713EA3E46FE34360F6A753147BA3FB51720A9174907E4EBB291F470FE3B3B2CD78EB05DD45D71F",
              "1": "D0D614DA94CC12D9415EA1B87CABE36FEB63A838D5161F4F1E5F477B097D039B3D7AF99E980DD70DF1556E09F2CF771DC129D638EADA84C7F80C04D0A3D1F45C",
              "2": "E3BCA383F3520C4A7B578DD1A05BB0207F82FD1B2FA95B55911051A8D6FE737E93833430C7B237F7EED783A973B8F67834C75EE09C3E20DDA2C02FC1C9D57C03",
              "3": "B605D3EA64AC6BC93E213DDB8043C1A97E91614FD97D84BDB9607806695E8A608D9A79025D9BABD87F7B18C8DBBC44D411404AEC7EF7A2EF676CBDE38821B616",
              "4": "E3EDC023F14F5196F1F3C63E54DB1025CA8D76CD81191EE19409484C897D199DCE896A4121296B223B19ACF2B3158061CEBDFA43C210C2E565C9156D72891270"
            }
          },
          {
            "RelativePath": "..\\..\\code\\tfDataIngest\\tfDataSetParquet.py",
            "Hash": "2A1123333AA05FE72A5ABAFD3D792A4793F6D97F7A2798FA4E85E854CD1A08F7B8D20CE9FCD9C1D1719AF77C9145DCEC3587C7CACD1AD497DF26FF7EE333ADC0"
          },
          {
            "RelativePath": "..\\..\\code\\tfDataIngest\\tfDataSetParquetAnnotateTrain.py",
            "Hash": "1E73A8FCF0B94A4A9F02883CA67C6343DEE46E3616AF3A3397B2CEF77AC54F3F565E3902590391A711956F9E074DEEF45FB80D286DFDAD21A59CFF738B327800"
          },
          {
            "RelativePath": "..\\..\\code\\models\\NASNetMobile.py",
            "Hash": "817396CA3E415CE9ACFFB2532DFBB1DF0C4A4FB68BA15A34E0DBA2FFCD821958795E5A97005C6FF86F72ED8A9F725331D8F829247B97B68233DEB2C6C93AF707"
          },
          {
            "RelativePath": "..\\..\\code\\tfMetrics\\macroAveragedRecallForLogits.py",
            "Hash": "390F7DAE9CD506008D2613E3683B8E9D065CE1A1A26FFCB5FA186457BEB13C08B54FAEE849618C91617119B816172E85A9D82B432243A3C17B6AE32C543605E0"
          }
        ],
        "Outputs": [
          {
            "RelativePath": "*\\",
            "Hash": {
              "0": "61F04E3150B112955061F4E9DF851DC1054806E76167961FF968FF7BA4F6E2AC2F39BC64CFADC7937C874004BC16E54B3AD162EE2DD489DD8383858DA769A0E7",
              "1": "9DD9AE7574B73D535B9E7E5A1581F8E99578E5697053A2B464D63A73073361F9E1C2FB4549085072F441CB8A0B0423AA46A1B71F024E4CAAF8E0DB83467122B7",
              "2": "64A50D588D40A706C0F40A9A836CCFBF428197288F81478D7B9063E4005C143E518E400D68574AF02D183D6635CC795DD4DACA1F3D463901DC9CD8C2208A79BE",
              "3": "83716EECE2C21C7A3F5D24B7EA2A97E1F8BF504EB9B7D6C28903747B259B7D8A151A61451631FF4553745D1AEBAB676DD31F54AE909143882F0B0938E94FEDC0",
              "4": "A8E6918DC84047CDF622C6B0E530D8D09B7D0F6151EA170FD7454D3A9A481605E61B993301C75E022A40D84020A4C5CAB04CB3523CA2C5B840ED93046F987144"
            }
          }
        ],
        "ResourceGroups": [
          "gpu"
        ],
        "SuccessfulExitCodes": [
          0
        ],
        "OutputIndex": 0,
        "WorkingDirectory": "..\\..",
        "Command": "python $in1 $in2 $in3 $in4 $out1",
        "Signature": "BF05913C5C3619384F19AB86DC010D8A7C4D85D7",
        "OutputsCleanDisabled": false
      }
    ]
  },
  "IsTracked": true
}

Note: input version has changed, so the outputs must be recalculated.

Saving all unsaved artefacts

An ability to save all of the modified (unsaved) artefact with a single command

Absence of artefact in storages and on disk is user error

Now it is represented as alpheus internal failure

Duplicate code in Angara execute methods

@dvoits suggest adding a virtual method called by AngaraGraphNode.Execute which has proper typed args and return value and it is async. For instance, this method already gets the index as an argument.

For directories with many files (~1 million) modification time checks take too long

That makes alpheus usage inconvenient.

Vector operation 3 not equal 2

See https://travis-ci.org/itislab/alpheus/builds/585648447?utm_source=github_status&utm_medium=notification

Microsoft.NETCore.App]

[xUnit.net 00:00:03.13]     ItisLab.Alpheus.Tests.Vector scenarios through API.Runs same method for multiple input files [FAIL]

Test run in progress.archiver: Archive content to extract to disk

archiver: __artefact__:__artefact__

/home/travis/build/itislab/alpheus/AlpheusUnitTests/bin/Debug/netcoreapp2.1/data/singleTimeDirs/ef5b4cbb-d726-4b9c-98f0-48b1097ee44c/dir1/test1.txt single file artefact restored

archiver: Archive content to extract to disk

archiver: __artefact__:__artefact__

archiver: Archive content to extract to disk

archiver: __artefact__:__artefact__

/home/travis/build/itislab/alpheus/AlpheusUnitTests/bin/Debug/netcoreapp2.1/data/singleTimeDirs/265970d6-b786-4f3a-a370-1114fe2f2a27/1_2.txt single file artefact restored

/home/travis/build/itislab/alpheus/AlpheusUnitTests/bin/Debug/netcoreapp2.1/data/singleTimeDirs/265970d6-b786-4f3a-a370-1114fe2f2a27/3.txt single file artefact restored

  X ItisLab.Alpheus.Tests.Vector scenarios through API.Runs same method for multiple input files [673ms]

  Error Message:

   Assert.Equal() Failure

Expected: 3

Actual:   2

  Stack Trace:

     at <StartupCode$AlpheusUnitTests>.$ApiTests.Vectors.Runs same method for multiple input [email protected](Unit _arg7) in /home/travis/build/itislab/alpheus/AlpheusUnitTests/ApiTests.Vectors.fs:line 72

   at Microsoft.FSharp.Control.AsyncPrimitives.CallThenInvokeNoHijackCheck[a,b](AsyncActivation`1 ctxt, FSharpFunc`2 userCode, b result1)

   at [email protected](AsyncActivation`1 ctxt)

   at Microsoft.FSharp.Control.Trampoline.Execute(FSharpFunc`2 firstAction)

--- End of stack trace from previous location where exception was thrown ---

--- End of stack trace from previous location where exception was thrown ---

Reduce operation fails if the index from prev run does not exist on disk and storages

Reduce opertaion (* -> single artefact) fails with

Internal error occurred (please report it at https://github.com/itislab/alpheus/issues): Failed to compute the artefacts:
        System.Exception: UserError
  ""C:\ML\covid19-global-forecasting\SEIR_single_peak_locations\ts_prediction\[email protected]":C0DA2F is not found in any registered storages"
   at [email protected](FSharpResult`2 res) in C:\repos\alpheus\AlpheusCore\API.fs:line 352
   at [email protected](FSharpResult`2[] _arg2) in C:\repos\alpheus\AlpheusCore\API.fs:line 354
   at Microsoft.FSharp.Control.AsyncPrimitives.CallThenInvokeNoHijackCheck[a,b](AsyncActivation`1 ctxt, FSharpFunc`2 userCode, b result1) in E:\A\_work\130\s\src\fsharp\FSharp.Core\async.fs:line 417
   at Microsoft.FSharp.Control.Trampoline.Execute(FSharpFunc`2 firstAction) in E:\A\_work\130\s\src\fsharp\FSharp.Core\async.fs:line 109
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.FSharp.Control.AsyncResult`1.Commit() in E:\A\_work\130\s\src\fsharp\FSharp.Core\async.fs:line 349
   at Microsoft.FSharp.Control.AsyncPrimitives.RunSynchronouslyInCurrentThread[a](CancellationToken cancellationToken, FSharpAsync`1 computation) in E:\A\_work\130\s\src\fsharp\FSharp.Core\async.fs:line 870
   at Microsoft.FSharp.Control.AsyncPrimitives.RunSynchronously[T](CancellationToken cancellationToken, FSharpAsync`1 computation, FSharpOption`1 timeout) in E:\A\_work\130\s\src\fsharp\FSharp.Core\async.fs:line 890
   at Microsoft.FSharp.Control.FSharpAsync.RunSynchronously[T](FSharpAsync`1 computation, FSharpOption`1 timeout, FSharpOption`1 cancellationToken) in E:\A\_work\130\s\src\fsharp\FSharp.Core\async.fs:line 1151
   at ItisLab.Alpheus.ComputationGraph.CommandMethod.Execute(FSharpList`1 inputs, FSharpOption`1 _arg1) in C:\repos\alpheus\AlpheusCore\ComputationGraph.fs:line 112
   at Angara.Execution.Runtime`1.buildEvaluation(m v, FSharpList`1 index, UInt64 time, State`2 state, CancellationTokenSource cts, Boolean doContinue, Unit unitVar0)

As Beijing@China index existed on previous executions, but does not exist in current run.

Alph file

{
  "FileFormatVersion": 1,
  "Origin": {
    "Case": "CommandOrigin",
    "Fields": [
      {
        "Inputs": [
          {
            "RelativePath": "..\\code\\gather.R",
            "Hash": "0B0F0669B78DCD50AE76E2F6FAD5D639FAB7DDF31FED0BBE422B04A69FD2AB6910CAC65E61B5688A85B2CDC3195382A95D8CF9DE5A87BB2CFCC7BD48639497A3"
          },
          {
            "RelativePath": "ts_prediction\\*.csv",
            "Hash": {
              "Afghanistan": "93101B1BD2B85C13FA3B2CE22B8B909252641A443E81C6B23EEEBD3E18DE2DEA4D7B2B0AFAA397A48F326F81E17CC42061E68CFB085420934C9BD2FB12B377C0",
              "Albania": "853E40598FF2C76A0AF33C87AD651DED21E8297C2FC54640B4659C035BC3B264B0C73B32F2EA3B524631DB754C409DC17538794B987F67074E1FF32A650634FA",
              "Algeria": "BBACD2DA3092E7A55A2A4A3466CC150EE7FFF1C81BCFEFCB8C081A6A52ED204D6275C9705A409E4E34E88D24DF7F02FD6CC95B3C9D2B7D5AF835EB192B99C3BD",
              "Andorra": "1FCBC40339FDBEEC0052DE6B9835C826C7B5D26F4EF7FA6EA7A3AB8A04E86626422D18C1F8DD581E5D215AC82F3C90B60D662D40CF512B6D810176A52BEEF679",
              "Anhui@China": "18D1B3007276C15C793CA9379E5640D44096AA2EC38ECA8B1077A80BF90C26D1CDBFCB7F04DDFEE42BF4DA7EA777AAFF54B41A3548C04230DD68313B1F483028",
              "Argentina": "03455A5C2F4096D78E2D91AAE74DE94DACB8379D10A26D25E457CC41499B266951364AA5B7225EE112ACE5DDD3969F0196927B91B26BF84873F0B772340E1688",
              "Armenia": "3116C672C5ADB6ADAFF1CCE54E44A7CFDB5CD06CD9DA9EA6C4C53773E6D358682A9762A86AA3D6C6755A551590A41C0FAD62DC5673DEB88F20730C00B44D748D",
              "Aruba@Netherlands": "74254BE45225DBB51B11B48E27D13D8D9091612C497CE6E6C0CA244E4C5EBD048C4BB580FD8EDB2A39609B479586372E98BF1676CE81712440EE020151A70484",
              "Australian_Capital_Territory@Australia": "455964955FA8712AEC3E9BECA2399AF296D5E28B3EC34CA419C49535998EC49B749B0699F7CC3E99ABC7C435279D4A6FD33CB4D744C3F142876791B7C6B3562A",
              "Austria": "0F9A2FA1FF6948D6A7A3C8698EE85811B840D3FD8FAD89ED6A3212F2DC0061AE760B3B90158458BD46A2F146150E07F8C16416432EB6D4787CEC9CB529403BB2",
              "Azerbaijan": "653D80BCE9F52CCFCCE633F47C1163310698AF1A2204ECE2A0A101D94D2B4861079B7E2071C77402E63858692028A31568EAC4C258140ABB101A85A6A20DB166",
              "Bahrain": "06A18FF806E98E05F4992356D6245CE52169AC93F0AFABDDB25A7EB6BB43D7CE927EBAE068CC3FCC3FF459CEA7DE6212D76351E2A2DDD03531BB300DD87B69C5",
              "Bangladesh": "B1FB50E5EBCDE6D73A67AFEA293BCA3E3DF75F1C5194D27EE894F8F4A2B19CFC3BEC0CE2E54F1B9F4C367A5C636A9E163CEAA1A1BE767E55B1FCB7E2FF8F18B7",
              "Beijing@China": "C0DA2F8051476EBC3892255AF7515BA8BC504141A671347843CE954630EFCB70E6FDB962FA9CE09239809520D003120F51B97CA202894D402BD5ED55460B2AB6",
              "Belarus": "4F9359AC666A5D811F24217E93A14BD09C13B8D74356C535F00CF3B1AA3315DAE6062026CCDA50E11D42434FB8D9CFEE03876FE1B3BD89F510E8D0291EAA5A9D",
              "Belgium": "202F8667DC7A8252C3B18B819390FADA64F68770E5959A86050E426FAE46DECCFC1D1EE1C8B440A3D72BC653D53CC4847D6C7B5FC472E00EEB6F1215B71C844A",
              "Bolivia": "1C02BA1C68AD96793A37FF578B35296998D505356F3D8564E097758183DFC6CED05E00E1D97D70518FBF8D40CC141FB9C4CAD58AD82B030A2579DBC0AA059489",
              "Bosnia_and_Herzegovina": "D01C2458CB355D6F748D86A20DFC354E595FE926FE7471B42608BC7546D13AD7AAE8E04116460FF200B664F6B2D06C9096DE85F0C86A0FEFFD81B58E1915B67B",
              "Brazil": "0B5C7A7E278471B7592063FE544E8B55428869830314919B4114D2A407052589A70700D2699B95A970982072305CF484F637A5CDA543EBFD7C2383E287F094E5",
              "Brunei": "1D0B075FA20135B86D6C37720E9C5A5DFD8FE6A27A65A74BBAAB8F4252788834B149F36BAE065C092AD9A76CFA60892D2476463014B5D05A806CF9A9AD2FDC15",
              "Bulgaria": "5A9BC9FA0D76EAE3EDBA7C7A55D73E42D9BA787211475DC7A34CD240091DC2650AD049ECDB8FAD5393BAA1C8FEF08B52D46F9FD180B26DA1FD419833F1B71587",
              "Burkina_Faso": "F1FDD1DBA506220203D27A7323D53007F7A557F1E584DDF01F2283EFA09FC75C67FD0F1E7F31592F7C26E7FBF5CEF3D207A902FAF2296A94E2099D183F2E67E3",
              "Cambodia": "198983379F95D895FB38C53514A1A6C6FB0E33893C8E1074892E6EFCFD3D769A0A7469011C9AB52BFC7B60C8D00E0FD576547555C22FC20B975EBBF734576761",
              "Cameroon": "9574D6F558C96140FF2B13DE11773EEC03CE3CE9BB2899A3716FCD3C6BC9C24F504ACACBE5FE265EB722B19F8E4B549D0AEE8C7051CC77134E73712DCE89F9C5",
              "Channel_Islands@United_Kingdom": "9BC6210B6A06B821E19C5EDD45FF021B777F7A66D9BFD10D98299CE744DFC88A113C15C6512F845F91DF40508E1FBEDD31D4A311EE10238C0C386221ECA450DD",
              "Chile": "2DBD7E45F095D9D3569CA88A4800C994CDB4D2520D1AC34410524A686AC885CD6FA579A45BC4CDDC8C70DEA0A6222B684BCBE7A55FD0CA23A52B31DE0C929511",
              "Chongqing@China": "2B41092D37BBB66CB8E636AA9FE0788FF5283F8515729C053A39B398A542BD0EB96B4495538AB8624555B3FB2EC7AA50C27BCED91B42DDBFFEA41DE61644BBBE",
              "Colombia": "6C92DE98379FBDA9AE78A84CF7E63DDD8F4821265A4E703EE9AF476476856C2E6E9CD576D4E20A9D7C5B6E7B2E5ED2BC5B5B357C048F9593AFE1298A96CED5DC",
              "Congo_(Kinshasa)": "C8235B09AD0876309EBFBBF32A902B0F01A587E76B5F6B83CC99CFDF02F75BE0A844EE52725DE1BA41B1FFA2E54E28490618DB7B3533E9DA1A3E925758E443AD",
              "Costa_Rica": "9BFD45B3795AC2768F8011838167EA460B41A4CAC23E4359A36834CB1D5D7C9E88E049DC3529EDCCD4CC42663954A306DFB7299AA110A08D1EFD1778E01729D8",
              "Cote_d'Ivoire": "52C6219227B7B305AE4CC629CF160DDD23A935E980ABE17E7817C16B6903C6C00D8D4D84C8D1BE9D8E7E32B8097F68D94E55B82F56B4E11D3B37F820E128095B",
              "Croatia": "9DEFE2BEE8F6C4587813CC60750BD0242F7DBC39CC200265EF675F6513F2B2ED95C9EB97F66BECF19133D21269F6DA65A483D1E015207906D7710CFE748C3E65",
              "Cuba": "9AC1D9E3865200DD07D0BB465482F147048313A3850A0A2EF15BF86977937EF814F270BF09974878088BD3DA428BC21A5FC2DC5F88FDFAFF5200F84F440C1FE2",
              "Cyprus": "BB05981BF08E5E0F6AF6BD778CB17A5F20FFC4C1C607ED9916BE6F1094FDD10EFBB62743EA08AF7500B4740FCDAF28A23F82B2F777BCB20D6A37F25A354731DA",
              "Czechia": "5661D41CCAC4BE6AC7719F88FDF89333F418A78CE35410E48459984DE018E87C67DC68F227BF03A87167E0FE9B2F3EA46A282D921B906DE940F6AFFD7ACF859D",
              "Denmark": "6FA3205A03CDA4B4A533BA4134A009D9BEE37144E918DBD0839AFEFA2F7BA603F7C1982AD655F7ACB058E5FD2E0163CFBD928F0E24CD6477FD90CF2D16CCFCDB",
              "Diamond_Princess": "49CDD0C55229F01E3F0D2F6C02A59092BD0BA78D5B47DEF3C0FCBBB07F1910387385C541081ED3B9327A092322A45E56573E237005F2159C6B7A4F1A7E692E89",
              "Dominican_Republic": "76EEF6C13B50181B2990216E33625912197CFAE8659363CDF049EE5CD1BB6EA3059FDFDE6908E80B9314E404E4262BA149FE2FF87D1C25B2E51D869491B6A839",
              "Ecuador": "5073FDDDED7DB7EB03E7ABD1F80563E53C32EB49DCA747EFD84AB3928FB9C8F6887FCFFE57C9E015E2F516B91C8B720D8C7D87AF00DFC14841876571B9D987D5",
              "Egypt": "871097A73F7A6D932865EBD32149B782CB6D36240BB9032ADD04E183C87079974CD6905DFC81EC5FA15581CCD842F168CF6959CDC85551DFB94C28BF18BDC525",
              "Estonia": "B04DBF41328DF8439A168A04FD77A550C99DF9B4A964197C263900E9A63195B4813604FF1C87D87A1F8FF7BB881AFF424F87B4467435BB951F1B72FEB8EA5AB6",
              "Faroe_Islands@Denmark": "5396C523FE7FB05C55D6D046E58CEF49A821D95E955AA7A5027285FBBD81AE30F7DA433616798298D3924405D3447F60B0C9AB513CBD94950E6F255928E16A9C",
              "Finland": "E26F26F8460BD0E552598924C26F01FA83DE38645574B6564248BBF455BE9D9EA23FDF84B5937F40EF023FC5CD8913FE5FCDED4CBF392A9989F7320877C47195",
              "France": "C03CFEC6EF21ABF91125FAB0C1CAE1029D7842204A312108D30ABB958D5CA2AFA90011DF2DE8A4BBBDCA30E1346107B7D54E6A4E5F8F486629AE6EDB361CE922",
              "French_Guiana@France": "6E879594019B00D3FCE4A49BCE2BEBC7DB9A2003F13DED77B825ABB3F9D81315A170B1ED063C9058E6C6A7354B6B6BD5F0195AE5ECAE98C385ABCE7E6BC45C19",
              "Fujian@China": "90749139CEE16F832651E566A8993B3FC8016EFDC00E1E10CDE1E769E0D2A93A1EB1162BB0749BD10BAE4DF9255E3D8B5326D6FCF62D1E0A13F3E60AFA628E55",
              "Gansu@China": "3B0835DBCF60B12AB364BC735B5C6970DEE984C12BCC4CDCBA1741A2CA91A27B6EC111B7C455041EF5B03412BA3B552DFC7FE25E4024EE0C180C4A9514366182",
              "Georgia": "0B19F20D66763078B664BB9722EB69E849D8EA942AEADF24E0AE18A94FB1B1F28BD852ECB21C4D99F5FCAED17EFC502A4BF0F599D81DEF38BE6519C5B00C4E12",
              "Germany": "CD5F61E730DDC153DA0BD3B08570EBF487CB80557775C7E11B2288D8D6D15D77D7A8DCBC5BDAFC689848C2D8AA51ACBCFFAC66373393DC141988C456D103F671",
              "Ghana": "0CF186AEECE7503B1AE2B7F4A28F2AEFB408CDF33B7E470F3B4A5EF0B1BB32F367EB8CE54FC4C7F72C4473777397C5109E26F0BC51EF0771C7EEABE51094C643",
              "Gibraltar@United_Kingdom": "E7D429EB3F6B6B29A6C5C5CAECBDF52240C78A21623EEC3E5C6283ECD51DF1AAD40B7B991EB2358382F80B2ACD90ABF8A126F69AAD0E8852DCF0F3844E7ECB06",
              "Greece": "347D712E5EC17850D8F607A3C4C6220013A6D41740FCE05D5E45B5B2B8653107864F165858E1810B1B328AFA397FF72836E4297016A024FEB63CCC0B36A835D4",
              "Guadeloupe@France": "7EE725CA641470108F95C17F0319EC61B20AD1447BB641A41B8CA9A6BC77A56D411651110D9DFED155BE6C42C8C8F4CC6CCFD669417ADDB88172C1624E87C444",
              "Guangdong@China": "3B2C48F3892E80FEBD1C8501D36B22A819350F11772AD762420285E407131D4CE29966FE15DA4BA4972E5866CC063B3295F347A56D46AA8C9A9DC0E1DBE0D2F8",
              "Guangxi@China": "5E8E69C52804F83CA7149D02F1B6915CCE9548DE73F5D15453B87630509393F5F324734FE9A457F678D27EEC9A8AF7A71D5B7930755967FD64525EF78C87122A",
              "Guinea": "C0197A88444AC948E92D147DDD3458C30018248DADE61B7BA386E36D8B2C3D4BE47AE2050D7007C1A79D44E8D46841A42E7A70B32885857B69F2D52DEC8C4499",
              "Guizhou@China": "37D0C680F1592F7E46F218013565BBE1FA46C12DF7A36F2F699BA7185BE47D226695A426FD473EE6D4F915AE1C5652865A9DBD8FF060EB3F6C0C7621DEF4D785",
              "Hainan@China": "CCD5373141F95A8795F437DB40423B0CCB02A4419E1D58B8650395A611A54C4762817B441AC220CB382FEAE2E8749E503EEB57FAEAFF1631E69648BCBED5BBFB",
              "Hebei@China": "B5AF3DADF5105B66A1F32B09B7DE2B814525D5F16F7DE8B866E7E6F451B116B013B8143E27D27DF556AFA2FBA5C3CC277B3B770794C6D666EE0257136D9ED0AC",
              "Heilongjiang@China": "E96DAF20DD636BC24A6A642519560F8A000B4CF9B865F668D61B68BCC809C3D2667A9438C315E0BFD8C4E9F7FE54C05AB628DB3605BFB02BC7BA7E79D871FA32",
              "Henan@China": "EF01325C08EAF8299CC8268E3254400A561CEAA294FA13415EC3E372B7C5984F39FD3F342C33044ABA4B054CBA32530026DCE5FB8B6231C07E8034D90B616D42",
              "Honduras": "6075F1928D7EE3D60E253D4759510C6CD14796837450F5DA047AD55F417E68F48466514090FEAAC0E52CBC9F2EB67456C95DA7328D9AA81729A1DE4E6F8E51C4",
              "Hong_Kong@China": "6D301037EA4798AB73B3A2C02085ECE6B32CF112D51213EF2F0C9039FB0CD4BF4B45980A70A6C64C6E616B163756D0C506ACE83DA87B3EA02697B3744C412F96",
              "Hubei@China": "7186BE6C00EBD60FF94529EE161654FB5CE8F40B449B1A2FDECD45C19722835952D7957673BE12B23DE0A1E0C7E8ECC044152DCD74D83161561F67F460AC5081",
              "Hunan@China": "88A923A0F864D11BFDFBBBCA80B40BE24ED94431E7B5D04F8E1388A05D1EA755A6AA1BD28E62C7B9D68B9F3026E61DBEE5E6799316275C3C7C48EB28156D68F9",
              "Hungary": "74CFE5215670F20E215245E86D2548000A5FF2484D3E75456E2CD9FDD61A124F3A6C53DA8751DAE15F51E9B4633F04FE031FC7E6EA053FFA9387BFAE5EAA46E0",
              "Iceland": "8F679AD153D7804457B7AE37CF65E8C310A4918CFD6625853AAF2E806AB4077573A9C5D41823B3713A81ED68C665A5E7A120853AEAE9704786F4A34A18172D3B",
              "India": "DA22ABF9A60100ED70A96BE4D52B582D360BD0F3DC9C9D193C756C61E3C43FD6C86D8E05B8F1FAB29F2F3D03AC5C8F299CF3B0BB05F5E7927E8DC9CE47E7852F",
              "Indonesia": "78EAFBE0B6630A9B29B4823450525CCAFEA13BAF7673DA5F5711079BBDB31667AD7C8774C01570077976F0129006F0FCB2853ABA1169EAD60E8A2B4BA6E20772",
              "Inner_Mongolia@China": "AAE2BCF5FE7EEB56CE5117A3EBB49C415FE379B8E7B5165C771EB180295953184D39C96097288D7FC33A8AD093898914CA028E4AEB1C66733124665192EDE283",
              "Iran": "D52840481F5B4EDF025D6FAF17A05696C6FE1BEBB305252F08C44C2E6906C61F8584DD1393EAD06A7F2A1561105F38B123E7B39B726549632E97F3BED92A6D31",
              "Iraq": "D72B5092B9E5556906ACFA884CC7D395E12262C9C9D34A3AB841FB5F83FB66F8DEE419A515C3943D626FB40E2852FCDBA03E6A5FE531079FE4D8B07929A35D3D",
              "Ireland": "7F682B78D21C9BD885B02195C81F43190E2A78E9D16FE51BE40F8825667075FE40BE18BD9A607B54D362C5DB5F658023141AA3EEADD258E0949CE7C076B0DE4B",
              "Isle_of_Man@United_Kingdom": "419DB7C91F18020F84D344A15A5BB7F92D1A9F0454770C0CEAAB8E00AF5F9D5822739A1D9690840084FF5AD5DAD9AA862C2BFD4C0F3202AE288E2D33394E6EE2",
              "Israel": "1D33E0087F29AA8044F872B9BEF9952ABA9C822F4C1E6A6AB80B96CF9EF9A6E7DE4262D287E86EF6130E938CB8C5EF9882CF6916FE7AD979DF4E8EDC3D4B287C",
              "Italy": "4F8226A3EB4A47E5B5D7ED3218A4F181885EBD09D8E94BBB6B3BEA57C87961DD2D8F3E3716A6DBDF4DA10F652E646AF4E000F56DD8C11A3281CDAE9FCA3D8BB7",
              "Japan": "C4E7F9216D4D7EA5255869F578D8595D4C8238611511174D5160728DA5FFC73D017A96E7D9A82103F692F36A904A0B0C40C31C1F615DFE9A81DCC6259190DE44",
              "Jiangsu@China": "3062AFD38D66EFADCDE234C492488C1E0A28B4C16F79B224E7B909E49901ACFF0857A9EBEA558655DFA7232758F11B9698E607001345278F8FA1CD0982C0D267",
              "Jiangxi@China": "CFA976DFDD6A90AFD950CFC0F738DE77F4EA598D281A03CBE983DFDEF5930974962B8604AF148AAF2D90AD128548695821B0E24DB9C1926F8D31F1DBD0219C05",
              "Jilin@China": "253BF73249A2FA4FFC3357945051418E9CCA7CB877FF37F39DFBAEFB366BE5F0A37012B9E998EE7B4D572B142CC721ABB65F0863B87FB76B7FAEBE9BBC66D3DD",
              "Jordan": "DD022470A2A40A9E7254A7F1F8B0BB11A94C462353C1697D476602F84277D9FF001487D2751DFA16855AE5B8F17312145B2E1049AC6784C85B78963422C6CC1D",
              "Kazakhstan": "C77A7059DD89C460C6BF8EEFD00EB0581CCFFBC9A485E94013D53FAFCF29D22CE3566E2AB6A4B1FB28826D9C691CFA30BFD76FDA1B8AFCE98AD3E572ECEC2F2D",
              "Kenya": "A88A50CEFE5E83352EC313FBAAF4521F3A78BBF908509355B15387E9547126EAB057D53F2A5BE1570FE33DD8B08A3DBB344E679A4D999350C8C05C7F983CB6BA",
              "Korea,_South": "6F66CD8415631CC4B385B2E35766C45251AE859A94BE383D09F499E4754AE878F82D37B22F3E7F9FCF9E0CF41B334ABADE8024E56196652CB7859B845ECF926E",
              "Kosovo": "5B34CE9CE25032455EDEE8C93C053B4882921C67EAFE903A887845DC60610E6B61C8BA08570CC2E323CF5F1FF4F75842CD02B0C1976B23B5AD81FD69A07FB494",
              "Kuwait": "F2736AE8621CAEBFA320AFFACBBFAD124C121350906C1F7E81A47F53B6619AC64D7E3BE1A7F7F3B8682E7D757F6AD6EDA0C33C9907D421C1234DD08EC076A402",
              "Kyrgyzstan": "4139E30CB7F9C6E2EE4BB8C8DA96487BC0C73708D1B063F1CB22D13F986395753252468B4400F2332E05C0EED28E1DC8A0856B82C086ED1338B991E138AB9A00",
              "Latvia": "5213D6DBCA1FDB21B4475DF35807BB22E0CF3CD691D84957F61A5DAE4A1198D4D6543B2CA3D2FF4B4A18D389DF5A7EED2CD454B102DD9410C97B76B5AB69CDFA",
              "Lebanon": "E393B6FE38ACF692971B9FDC6E4791AF7DED634CDE387C5808FA29CEB1356CFB4E914F2A951C86AD46339790919782E5BDFC8A5F3084C89E48CA8836C1F8DCBB",
              "Liaoning@China": "498C45B7896719AC0B02EE3815BC6A89E6EE4F97B9147AAD594C750C1607595C164DDC1E12C92D07ECD501CF307BBC0D77716551F2EB4BE834D966CD70F5B514",
              "Liechtenstein": "D60FA0940CF68A4FFAFBDB086A322F1DF1F2D633C8357DFA67639638E1327C9B25040456F54DEF6189FAD285B9B89ED534A357542BAE414650B8452838A50EDF",
              "Lithuania": "DF15A9C1291B63597BDB201729C393E4ACD8A325B535A8B0333CD4C48FCE5F524BA2E0175428A650E48BD26D2A016D20441556A22B36A82FC1AD35D8D053D99F",
              "Luxembourg": "D7CDF8A8756A4879E030F8453885434678EB8AABB49B6056C04EC5966B15E572B9D4E0754E4FE6D072879A0BCEDE6ABAE0FFB824CB0FDDBB923B419A06C7CD38",
              "Madagascar": "BFB65F8366F9D3FBDC2071872E00F5E80C2DC848BCB9EFE5C2B33BA59FD0815C0F360A56A46D9BA358F38C40739CB8024C4658E5F38CC017B54306B6098DD53D",
              "Malaysia": "E20BFFE9F0143CEDD7B798D022C6D9218D85886E4DACC9EE550580361C417CB65855095724E4F41A224A768E9DCEC74A0A034733F28519DDF2EFFDAE3458BA00",
              "Malta": "0167B08A654A736583692170F8A524D1787C4A03E69A10BDF9E0CB025197919CC2F6CA5248D2FF52F946797AA972EB3196AD1C7A5552E5E35A1A1CA6C43B3A83",
              "Martinique@France": "914887FD65DD0C78F637053918DF71365B7004D46BAD638B1B5861D7F53740A71EB8C6DF66F2D84A5F149EF6BAC40F168B70149E15944B7291FA804F11793345",
              "Mauritius": "804DDCEF4C83D24F1D54D8F6EA4CCE61303E79D759358DCB72FB7993BBE939A7FB252DEF49694ED2C5317B22A1DA50289BFD50ACEFA06447C9F80B54A941A0C9",
              "Mayotte@France": "BE24353F9DFE0829B12F15359537D5397D915B03D1D75380AE821B417273767A46944EC5AB45F09FAF8CEEBD170A62181654F606F409734154722399A64F4209",
              "Mexico": "E3DF094BD00CF9250C49F4A977CBD72FDA03F6D7C2153D091ED8AD1C6EFD4C684C4A0658A93F2DD3626E9E4F1ED6581921FB5943B50DD7BCB44FB2D49B5DC53F",
              "Moldova": "595B4527AB2EF966FA76BAEB639965F96DB547D9F95CE45942F29765C5DF83408E1F66C6BE42175F5DDE9ED78AE4F21DD59B7EA744C3BDA03AF0EA8B19359BFC",
              "Monaco": "F4D61296EA710338A3245079B839DAD2285F91AAA82DB47885D1978D816D79180760E54D7DB5869D21D5FEB64BA685D332FB81EE99EC1A4DBFBD4FEA095DB638",
              "Montenegro": "7B6487A5ABBC00B92C2659DF053D0E1D434B73A54B0F02081D7248340C9B4CABB1B7FE377590CC6E8EB207FE504A1B4F7A0080B58449E7D280A99DCF63626F75",
              "Morocco": "62FD652AB12C17C0A604CC852A86DF50F920138E4CE261B3AEF21C7DF38607C7D01C6CD957771CC1C2EF24A07F7C420F0AFBAC2D16242F254C58CD2AC03D09E2",
              "Netherlands": "7654FC4931C3FD24EF5E3F46A455C22C394415100C60FE7152DF38AD9C54F46238E71863D5C60B63147EC8CB015CED5AD1001862AADBA8A53A954DFAFBEC52E8",
              "New_South_Wales@Australia": "B696A5141289ACFC9AA2C0CCC8AF34D5157261D5B61224868B126872B42B89F7F79CBF3489536CDC4D9E2CDA0C87728E8A9F0595D7B37B754C37B85B9A0C5F84",
              "New_Zealand": "E7F0789D9ABF0C77EFEFED4F6A7B1A213C3DABF9E72CB179931CEAF805C54A0B4399105AD3CF377E4F87C5A7AAD415F4426872EB42F6557143B506FA56F596D1",
              "Niger": "AD55851D787260C95F7FED10C5E8B851DA1C8D0A78D94E73D50F86F6378A4A114BF8E2C35132637166C3311B0F2C3C6DA5A25BCD971992F6591083061ED01332",
              "Nigeria": "621EAD55018569F8C4997E6A97B52437C5A1C0091956571DA4A900350A3161BF1F7C9D18E19A362677308BF14969FE18B65E9911541861FE74CE0452793C27F5",
              "Ningxia@China": "1511AE9A71A8AB30BA7914E2FFD401158D272A057E3E845F67C940C16E14F94F782768449CA23F02AE65E420AC06EAE897138C90511E14173C2CA1C94B318626",
              "North_Macedonia": "798917B9E4A3F20FC0FE464034DC4912EDBEEEDF0E193D8840C3F23BB243D715DB7D2A0D247327203CA32FEBA298236BA3E1C4254133BA3629F36B3905BD926D",
              "Norway": "9A1B314E39AA08088DF52160D72FA4D60E88F504FBA58E0D00C2165875D103EF9B69CA315A553240E7D69BD933429EA6186F99DB7F984CCC277D0BE86D962E4D",
              "Oman": "9B42F7782E39C475040E3EBD3B7B2440903DA0FD6AB9180CC0FFAA3D751FEE1DB758DA2B965783157790BF9A317CC43458C7996BE11F0EE53BE395A47DB8B81D",
              "Pakistan": "7D2BA3D0E297C6289177CA4F03C8961024EDD9993D0B5182247CA7CB564D9AC92E1243F6FDCB2DF6E89009F817B062EED920285B6E51F35AD1098DCE1313778F",
              "Panama": "C5CBFB008EE4F8B045C3E6481E135114B72CDE3225318151094E54765EDC1D3670595BA2DCC2B1DBAC9C64B629E42FF6BA4B1B036E4F7BF3B7F2AAC697333C6C",
              "Paraguay": "6A088D54CB0DE1CBE8B075B71062D051C72760E98E971383A388DF0A99B4DBF39B44AB334B5EB4876A946DD60761F5E346939B501A0AB4CCD997573417290595",
              "Peru": "4845DA653A29FCD49D5278CC2AA1B4A62742F59A22A45FCC1ECADEACE578BDD62DE6123AAC8889501AC354726A9B46DCAAD7AE5D908F7AEC7B79465AA4F16296",
              "Philippines": "653EA9932FF6A2F5BE009FC1BAA5B5B5F90813A646EF1FD69C7351B70197329DD726F3AAC5359CD4DB0F16C78518073DB077B4449731FFC5A416605F8DA2FDC8",
              "Poland": "F8C44EA51645F945AC1FB06DE14C1CC9B2412A81FAD3688D50938DBDDFB969780CD9CB86567644DE54CB1F7A9069786EBC7B748726734F5CC0254CD42CF13832",
              "Portugal": "F12A65FC3D56D6B41473A3BA6D27D36E41384607985207224F85594ED31167116401998170328AE4003EFD7F31A21D82C5E01B47D0A929DAC6CD7497557CBE3A",
              "Qatar": "5317DE5941BFD14AD1AF8D06AC5A9D49EB9AB8456A285302FA2D1B456B51DB2DBD551BAA4929BA042599C78B836EA1505872205CC3A877F822AE7E19749C9862",
              "Queensland@Australia": "AD4AD6BB6EF127FA15639F7456973E5169EA68A4A38A8497B0DED86A220823E7D1E1E94A2363958D333A905A94043786518F38FE4BF1B64FDD8EE7B3FA222FFF",
              "Reunion@France": "0E4AEB8F0DFC15FFF41E521ACF60E65DF22A305B35F22494EA5AE7E8E510D6B42DEF40E5BB17F91CAD42603122F8153FE5AA8B8C39AEB83C9FFC4AEE2755E14B",
              "Romania": "1C361239FEE38DF49EEF8248385EB58AB73FA86366A92A33ED76F81396943D5DF12B6936E794CFFF5FE1E1594B9A89120AA78C1912F66F1B67851E1EA59D7A3E",
              "Russia": "C885B0548AD23430B70C9475786EDDC731626150E396F2E424EF4BB5D154334BB36C5EC3280F72A91D1B18DB465F7CE7B20DD7EDCEFB282F5C22B59A99066031",
              "Rwanda": "7473E3E618F0DD1BDA8A6691531FA065E5CAEED66611618A6ED985DBC40308C093DBB16EA1CBA874735534BB48D825B4A466064D9EA628C86CBE42808C43A0C1",
              "San_Marino": "C70C83FC62A48CD783346422E9204F2291E916599865E74A7D7A0770475904E3DBF6443BCC55E44A2725035C282454EDC8552FF08726F73CAC6E1EDE34CEFF71",
              "Saudi_Arabia": "375A5DF26DCEDEDBB8F17C0CF5CC0AABBE7C158E59ED022BA5B4B66A8DDB043836522536E1B0AF279B023E0C77DCB6A4420532531E37C4E927769CC6E751CE59",
              "Senegal": "53A66698D673FBF6071B74A49995DF7C8DFE03D78DCFEC7EB7DD1579357DD18799B3DC5E5A0BC03E7571F3EB1E61217AE384EF6CECF649201211090B53A550A6",
              "Serbia": "26E372FDA413E972FE870BA1F2F2088F2A9C34ADFD7BA4EC7316720B4C549688B5A374DB7F6324BF09D942CDBDCDE634FC1FDE0B6356D50883D6BB8811111B45",
              "Shaanxi@China": "F3228A90295F7B5B06ACD43180A6C0A5DA774A081A1749FB1885F18B0C191F1623E4B62D6F362801E977E61DD6212F00852CFCC437845E2FBB66E424C17FBD31",
              "Shandong@China": "F3C1FBF4086438591D7ED3F0D0899A59D5F0FA4E8AA63B64007BA6FCEA24BCB5E19745ED7D601B792C07CE3FB460EED4E0993D88A8D256D910E916AD7CF1BC0A",
              "Shanghai@China": "E7C8E8D0C5C6B9434347F6038BAC65F5BE89890222E3B1E1A91BBB1C18A0EC2A00CBF04CE8AEEE29E3358C1C71474BB6266E1A13D95CF5D85111A790D3C253F0",
              "Shanxi@China": "143AD2B9FBABB41B09EF8075202104AEF8CF7776335532D2F9230A1263AC1C780283EF7687084798AE35C76AE537ECC6A30293FA72F4D3A898E0F6F1EAAC9BC7",
              "Sichuan@China": "1551B936838893682FB6575E2D67E338F9BF3B5EA0D3B6FF5356318E2079DA4AC73DC52BA09FE1911508756FA95A68DE4C70AC07D79A0740425E93BF5834B655",
              "Singapore": "98F51C07D65E2E0B988F40A5748549326C58E2DA100D972440C98387BB8CA55CB23E4B7770FD6910BE535828E03A133EC144C388BCD87BADD2FE5611682267FB",
              "Slovakia": "5EEDDE9DC6999DF5DBF0DCDABA212D3E9A5B454E0E1E4852AD1ECBF0DF89512529C393B32BFB0AA5E46094F73A16E9C507D172067CAB452BE5B92143E73F0601",
              "Slovenia": "F8569ECB8F90DF68A51276A32E44FE54ABD761A54F8E68285734E1F6279290D6DB642BA0E4475335E6212E2A2D7580AE038B6D20A8FC8CA0852AA19BA3904333",
              "South_Africa": "961D8F1A3998B2665B558F89AFE6D4683CD78B098A700BF54B06E821664210B88E72348D2FA1DA9B69FDFF7A7BAEC5E132EC0BC29D2B3D837E74DFDF4BEABCE5",
              "South_Australia@Australia": "D0972A87F16F908A253FC2C0CEC010E0E8ACD83B4586409D75262D9F64E4E446DB7EF2FCBBAFE75D7BEA16F4C765D7CF28C347FBF6FA0DC6CB227343BBD2C8F6",
              "Spain": "AC8B46CF01540F0F01BA17560A1AC1371913E8789C64ADA6A8C9E776628DE428F12BA7A36F780CFD93ED29FC294BD6CFA2A46F8F291E1154FB0FE9CA34186CAE",
              "Sri_Lanka": "667A94E9CECB6C2B1D91EEE1B44EE27D7E1B55440DB401206E5FD80E31FF0654585BDDB0F6318DEF2064CB1BCA7E90258AC5DB864B71293C976D9E5ACD4FEBEB",
              "Sweden": "4919DF9DF12760C01991B1DD7F56D8091EF0B7D2B9B61DC15A589306CC6B69321569A9CD795EEAA600F99D66ACD4EABCB8EC0F77D396543AC8D4F1F6341AFED9",
              "Switzerland": "AD8D5F4F93EAE26F0FA9785662EC9F1C8682EDAB4794204110A68A44C0101F39B0AF1096C9D5CCB5F08355640296E9CB32A1949D64FAFD1E271CD9FEAB5719F0",
              "Taiwan#": "78E4FA0766BCD31430A86E23BE56FF735D7C94C7CBDB53BC2076B8366484A8AE49BB96AB349ECC9A4D5FE99D339143F74473369B6EF04859E41C42C6E4F81995",
              "Tasmania@Australia": "F5FF40530DA83981238997A9F6739D42B3894DAC94A4C38709F8D6EECF0A73DBC197F9F9AC73DF3AACBCE0E1449F3088E40E68C7B4D5E95A5D80D13BEA14E1DD",
              "Thailand": "102591ADD4CFB62003A922E8ED5B4B3E9E65CFA74748A83EDDC8EC120FC94CC8088234F5AA12B66F5EE07E512D0ED8ED11081917C7053A7C7E329B5300A7DB18",
              "Tianjin@China": "8D6A40270F12D1931B9C209C384DD52F1049506EB17A90C60938057D314867755E6D1B4F93CDF184E5E04C53A133C497F139515E0198618976702C9FA5A46FAD",
              "Trinidad_and_Tobago": "4D37C20D805C168902540CC7664364C411F36CC0839C8858D2041E607EFAF13E65D744F845C88591253EE984FDC2E1C12BE79670A8F0EC86B8FBD4CDDA4C6790",
              "Tunisia": "C4402F4D51C309450E47CF9CD5D53EC5FD13CA91CF23CB576FFBE886CF05CF697E28E68EB328AE3F8C48606DC755D88A9A93483D8B8CF7BA76A20E213AD6F621",
              "Turkey": "1D629CCF065CEB359E513C0328CFD23C3A038039CF681D5ABD3D7BE65DECBCAEE1FF3DC4326028092CA2DC579134078E5E56CB5A8BC5B59E6FABDDAC391C58E6",
              "US": "49FF3BB8FA394F1E02530BF828117D242714F9704D911B4615FB790342C7293246FC3A72D91C2971E7ECC9719E230642CC564732141E90B1512C12261D303C3A",
              "Ukraine": "4E6AB55478BED38151D7E937C273DAB49F539F2344AFF729D8437C6D29364692156CFB0AB048371F9C4251B84AD707408A3CA4A7904F948277A3B0B659816E52",
              "United_Arab_Emirates": "F3BE4C70DC42536F4AA74EB9FA208FE3FFEB3C2B18A39C8F6048A70F949C78A80FE4887F0B7112D30C4B2F2BD5D0CD56107C396011BB06FBF88F2E69FE81621A",
              "United_Kingdom": "329538E511ECDD5B7943BA424B29FDFB7286079D62D0A5C11790835A5E52D741264AA146AD9C1C7EC0C1806205591212F8D7C1B4AFF8D4E99B161678F13B2C42",
              "Uruguay": "541312122EB2AF11B3D9BDDC25747AAAE6749A3A166801F9393C030241B5C4AA9E462E8BC82F7E1F97339B8F3952E49276E5C9F8A7FEEA71E08DF75478D64B17",
              "Uzbekistan": "660D96EE8F4F9BFB8095A4878E2B2976025CF38A7F9BD654307B65DD7ECBE577799DDC9F6251AC6DCB216E12A2047C695FC6E945BDFBAD9A71139903BB0C74E6",
              "Venezuela": "11848B42BC6036815DAC3E37061E40A5F777D077DA5A4AE20F3B3B0D30017A268F379E31A627C2CA731574EBEA53932151300FE2C7DD5F256BD795845F39FC3A",
              "Victoria@Australia": "41DA9C36C5C0763D9AAC8C4BE5E73B2BBFAD1BEA3FE465ECE7A46220A64A7E91B43C959471A1F85936AB8656A8BEE633C78CF4494E0FD8B5462D467625CCCD0A",
              "Vietnam": "01D4E5C8C52262C70A263C42340AE6D7AEFAD67C03D7386BC928338B83EEDEA72D41D28D5FA0F269190923BD763A5B385362C53F3C2AA6BEB49357BACAFBC2B4",
              "Western_Australia@Australia": "A4F58B363D7F12050AD22E4EBF1AFF2CBA450825E5F1CF90D92F8A5C5A3F498C46D9B706529E0A13E4FA6300226309FA6F9852F7DE1A2C051D260975F9298AD2",
              "Xinjiang@China": "3BE8570B131DA80A6AF716B322FBEFE27ED21CD176FC17AB7D32831F7FDDE19855870A2B14915DE68418A24B214E2946E5EB7636A47C5F9AD5DDFC0475CD695F",
              "Yunnan@China": "547115B19501A9A3B754CB19D24DBB634E57994F836C80C14F9B7D4055F5465EB8A7689E18938F39A165F3ADA64C38EA312F802CC0B6574D56E38EC82AB3CEC5",
              "Zhejiang@China": "A9C3B276F335B00DA67293CCB689FD9C79515B66C821E782F28C4C359D206205230F727D7F09CC4E7A359F2D735DC90819CBF60DA480C6DAC14F3B0153415544"
            }
          }
        ],
        "Outputs": [
          {
            "RelativePath": "ts_predictions_gathered\\",
            "Hash": "8C1AF32DAAA95F8BF8831FD2720695FBE515F94EAD1A85251E8243B33EF2FC12B289817CEED2E707C476D9B65B4B5845D4F4EA141E258A7F57E10DED4063C3A9"
          }
        ],
        "ResourceGroups": [],
        "SuccessfulExitCodes": [
          0
        ],
        "OutputIndex": 0,
        "WorkingDirectory": "..",
        "Command": "RScript $in1 '$in2' $out1",
        "Signature": "2A07DF993DE2003C04565DBED1DFE8CA00F2B166",
        "OutputsCleanDisabled": false
      }
    ]
  },
  "IsTracked": false
}

"alpheus help" prints an exception

Run alpheus compute help.

Expected: text describing the command and its arguments.

Observed: The text looks like an Exception.ToString():

Argu.ArguParseException: USAGE: alpheus compute [help] <File or Dir>

FILE:

    <File or Dir>         File to produce/(re)calculate

OPTIONS:

    help                  display this list of options.

   at Argu.ExceptionExiter.Argu-IExiter-Exit[a](String msg, ErrorCode errorCode)
   at Argu.ArgumentParser`1.ParseCommandLine(FSharpOption`1 inputs, FSharpOption`1 ignoreMissing, FSharpOption`1 ignoreUnrecognized, FSharpOption`1 raiseOnUsage)
   at ItisLab.Alpheus.Program.main(String[] argv) in C:\data\sources\github\alpheus\AlpheusCore\Program.fs:line 60

Support for scatter/reduce operations with meaningful error messages

Alpheus doesn't track implicit dependency on sub-item of an output folder

Example:

[1] build -o "files/" "download" 
[2] build -o "metadata.txt" -d "files/data.csv"

The method [2] must run after [1] since if depends on a file of the files folder, therefore there must be an implicit (non-user defined) dependency of [2] on [1].

For instance, the source method for "files/data.csv" must have a "structural" dependency on "files", i.e. affect the execution order.

This should also apply to vector operation. Being enabled, this allows implicit reduce/scatter operations, e.g.

[1] build -o "files/" "download" 
[2] build -o "metadata/*.txt" -d "files/*.csv"
[3] build -o "summary.txt" -d "metadata/"

instead of current syntax:

[1] build -o "files/*.csv" "download" 
[2] build -o "metadata/*.txt" -d "files/*.csv"
[3] build -o "summary.txt" -d "metadata/*.txt"

Enable synchronization of methods on resources (shared or exclusive)

exception during load vector alph file

vector element [4] exists
other vector elements are absent

got error

(venv-gpu) C:\ML\bengaliai-cv19>alpheus compute experiment_outputs\dgrechka_1_mobileNetV2_bottleneck\vector.alph
Internal error occurred (please report it at https://github.com/itislab/alpheus/issues): Failed to compute the artefacts:
        System.ArgumentException: Rank of artefact is 1 while index length is 0
Parameter name: index
   at ItisLab.Alpheus.ActualArtefactVersion.checkRank(FSharpList`1 index) in C:\repos\alpheus\AlpheusCore\ActualVersion.fs:line 18
   at ItisLab.Alpheus.ActualArtefactVersion.Get(FSharpList`1 index) in C:\repos\alpheus\AlpheusCore\ActualVersion.fs:line 43
   at ItisLab.Alpheus.AngaraGraphCommon.getActualForExpected@119-4.Invoke(Unit unitVar) in C:\repos\alpheus\AlpheusCore\AngaraGraphCommon.fs:line 119
   at Microsoft.FSharp.Control.AsyncPrimitives.CallThenInvoke[T,TResult](AsyncActivation`1 ctxt, TResult result1, FSharpFunc`2 part2) in E:\A\_work\130\s\src\fsharp\FSharp.Core\async.fs:line 398
   at Microsoft.FSharp.Control.Trampoline.Execute(FSharpFunc`2 firstAction) in E:\A\_work\130\s\src\fsharp\FSharp.Core\async.fs:line 109
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.FSharp.Control.AsyncResult`1.Commit() in E:\A\_work\130\s\src\fsharp\FSharp.Core\async.fs:line 349
   at Microsoft.FSharp.Control.AsyncPrimitives.RunSynchronouslyInCurrentThread[a](CancellationToken cancellationToken, FSharpAsync`1 computation) in E:\A\_work\130\s\src\fsharp\FSharp.Core\async.fs:line 870
   at Microsoft.FSharp.Control.AsyncPrimitives.RunSynchronously[T](CancellationToken cancellationToken, FSharpAsync`1 computation, FSharpOption`1 timeout) in E:\A\_work\130\s\src\fsharp\FSharp.Core\async.fs:line 890
   at Microsoft.FSharp.Control.FSharpAsync.RunSynchronously[T](FSharpAsync`1 computation, FSharpOption`1 timeout, FSharpOption`1 cancellationToken) in E:\A\_work\130\s\src\fsharp\FSharp.Core\async.fs:line 1151
   at Angara.Execution.Runtime`1.buildEvaluation(m v, FSharpList`1 index, UInt64 time, State`2 state, CancellationTokenSource cts, Boolean doContinue, Unit unitVar0)

Vertex command:

(venv-gpu) C:\ML\bengaliai-cv19>alpheus -v verbose build -rg gpu -d code\dgrechka\train_mobileNetV2_bottleneck.py -d data\bengaliai-cv19\ -d data\5foldCvSplits\*.val_ids.csv -o experiment_outputs\dgrechka_1_mobileNetV2_bottleneck\*\ "python $in1 $in2 $in3 $out1"

in storage up to date is shown as local up to date

dicom alph:

{
  "FileFormatVersion": 1,
  "Origin": {
    "Case": "SourceOrigin",
    "Fields": [
      {
        "RelativePath": "dicom\\",
        "Hash": null
      }
    ]
  },
  "IsTracked": true
}

structured_png_auto_contrast.alph :

{
  "FileFormatVersion": 1,
  "Origin": {
    "Case": "CommandOrigin",
    "Fields": [
      {
        "Inputs": [
          {
            "RelativePath": "..\\..\\..\\packages\\FSharp.Compiler.Tools\\tools\\fsiAnyCpu.exe",
            "Hash": "F08CBA03841C523C8FA81596E7A5D4B26B795A5F4DC2710C8279A70732B9C46A7E432F5BFAACE37DBA1EBD9844A4BF44105F20D91F50DCF48560C3342365F10C"
          },
          {
            "RelativePath": "..\\..\\..\\code\\data_processing\\dicomToGroupedPng.fsx",
            "Hash": "47FCB77C92540A63E627AA303BC31D931FC758F5FC7521CD813D1AD80E696C5FE5545EF6A71AF2983F5B2E79D1136B33009AF8C49BCEEE6DABBE84C0FCE1DB6F"
          },
          {
            "RelativePath": "dicom\\",
            "Hash": "05D6213BCD3E79743644837B51E33D4048BA2570F9B63064D3768B9CC7D381AAA38633080585A9C761C63974BE082F9A557F209B4416094FF7E91FD2D19AFFA0"
          },
          {
            "RelativePath": "dicom_metadata.tsv",
            "Hash": "F412EC0C79E4B040FB039B670528C7CF0750382E3CF11A4EC3202085EC8A9A2281C45845FD9074EB63D45A6271A146544DD90262FF5105FE6DD1533B2F1DAE8C"
          }
        ],
        "Outputs": [
          {
            "RelativePath": "structured_png_auto_contrast\\",
            "Hash": "B7F06AEBAFBC7F78B7E810EC05AFEAF2621CE86915010BF8B72F5DE60081F45B2F0C2D0379816A3509CB91D96569E66832A024412590ABDC387EA99ABA72753C"
          },
          {
            "RelativePath": "structured_png_auto_contrast_registry.tsv",
            "Hash": "9D3BFD86C29D22BCD86DB2B525B695E83707C4B2AC0FDDB5E494E02ACA9FA559297D7B777ED9EA94C4C80147CCE2E3C356FDEF77CDA64F20FAE892E71F9061E7"
          }
        ],
        "SuccessfulExitCodes": [
          0
        ],
        "OutputIndex": 0,
        "WorkingDirectory": "..\\..\\..",
        "Command": "$in1 $in2 $in3 $in4 $out1 $out2 auto",
        "Signature": "643EC03F47DC9C2A33694142E286536121F7A1A4",
        "OutputsCleanDisabled": false
      }
    ]
  },
  "IsTracked": true
}

Default storage

As a researcher I want to be able to setup a storage name used for saving the artefacts by default, so I do not need to specify the storage name (and recall it's name) each time I save the artefact.

Alpheus initializer must disable autocrlf for Windows

The problem is that by default windows git transforms line endings to unix-style when pushing, i.e. prevents cross-platform work with the repository, since hashes will be different on windows and unix platforms due to different line-endings.

".hash" file should be created right after artefact restore

Can't see the graph structure

There is no way to see the graph structure. E.g direct dependencies within artefacts shown with status

Last modification time checker does unnesesary job

Now last modification time check extract the newest modification time within the directory recursively. It is not needed to walk through the whole directory tree.
Actually it it needed to check whether there exist at least one file/dir that is newer than we expect. If we found at least one, the traversal can be stoped

"ItisLab.Alpheus.Tests.Vector scenarios.Scatter-scatter-vector-reduce-reduce" test failure

See https://travis-ci.org/itislab/alpheus/builds/640586896?utm_source=github_status&utm_medium=notification

CommandMethod's restoreFromStorage function must be thread-safe

restoreFromStorage is given in the type's ctor.

If it there is an artefact used as input for multiple methods, it is possible that restoreFromStorage is called from different threads at the same time for the same artefact.

The expected behavior is that it downloads the artefact only once, and the second call just waits until the first once succeeds. Note that if the first call fails, the second should throw same error.

.hash files are not deleted during clearing of previous runs output

Gitignore entries of the newly created expereiment folder

.hash files and local default storage must be ignored by .gitignore during experiment folder initialization

e.g.

**/*.hash
.alpheus/storage

Composite extensions are not supported (e.g. *.tar.gz)

Expected User error, as script did not produce the output files (scatter method)
Got

        Newtonsoft.Json.JsonWriterException: Token PropertyName in state Property would result in an invalid JSON object. Path 'Origin.Fields[0].Outputs[0].Hash'.
   at Newtonsoft.Json.JsonWriter.AutoComplete(JsonToken tokenBeingWritten)
   at Newtonsoft.Json.JsonTextWriter.WritePropertyName(String name)
   at [email protected](Tuple`2 tupledArg) in C:\repos\alpheus\AlpheusCore\CustomSerializers.fs:line 24
   at Microsoft.FSharp.Collections.SeqModule.Iterate[T](FSharpFunc`2 action, IEnumerable`1 source) in E:\A\_work\130\s\src\fsharp\FSharp.Core\seq.fs:line 495
   at ItisLab.Alpheus.CustomSerializers.writeValue@13(JsonWriter tupledArg0, JsonSerializer tupledArg1, MdMap`2 value) in C:\repos\alpheus\AlpheusCore\CustomSerializers.fs:line 25
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeConvertable(JsonWriter writer, JsonConverter converter, Object value, JsonContract contract, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeList(JsonWriter writer, IEnumerable values, JsonArrayContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty)```

Test "Consistency across alpheus versions and platforms" fails on local Windows machine

Test output:

Test Name:	ItisLab.Alpheus.Tests.Hashing.Consistency across alpheus versions and platforms
Test FullName:	ItisLab.Alpheus.Tests.Hashing.Consistency across alpheus versions and platforms
Test Source:	C:\git\itislab\alpheus\AlpheusUnitTests\HashingTests.fs : line 86
Test Outcome:	Failed
Test Duration:	0:00:00,023

Result StackTrace:	
at ItisLab.Alpheus.Tests.Hashing.Consistency across alpheus versions and [email protected](Byte[] _arg1) in C:\git\itislab\alpheus\AlpheusUnitTests\HashingTests.fs:line 93
   at Microsoft.FSharp.Control.AsyncPrimitives.CallThenInvokeNoHijackCheck[a,b](AsyncActivation`1 ctxt, FSharpFunc`2 userCode, b result1)
   at [email protected](AsyncActivation`1 ctxt) in C:\git\itislab\alpheus\AlpheusCore\Hash.fs:line 39
   at [email protected](AsyncActivation`1 ctxt)
   at Microsoft.FSharp.Control.Trampoline.Execute(FSharpFunc`2 firstAction)
--- End of stack trace from previous location where exception was thrown ---
--- End of stack trace from previous location where exception was thrown ---
Result Message:	
Assert.Equal() Failure
          ↓ (pos 0)
Expected: 5079DAD9B4A4565053F1FA772E2E0012500A510A4···
Actual:   3638496D6DDE8A6F0B7D6DE2E285B872AFD54FE8E···
          ↑ (pos 0)

Executed on Windows 10 in Visual Studio 2019.

Possible reason: git checkout automatically transforms line endings to Windows-style, so actual hash for the checked out file is different.

Vector artefacts are not saved to storage

Successful non-zero exit codes

Some of the programs (e.g. robocopy, rsync, etc.) can return execution summary as exit code bits.

Thus some non-zero exit codes can actually be successful operations.

Git ignore management

Saving an artefact with alpheus adds the artefact to the gitignore

Alpheus does not skip remote up to date vector elements during compute call

Consider a chain of vector artefacts:

A -> B -> C -> D

if we request to compute D, C is up to date on disk, and B is up to date (remote) the execution hangs.

After manual alpheus restore B compute command alpheus compute D starts to work

Only directly spawned processes are killed after ctrl+c

If spawned processes spawn other processes, those are not killed

Alpheus save of source artefact does not update alph file

Consider the situation when the source artefact data is updated.
The user then issues the alpheus save command to the artefact.
The disk version is saved to the storage.
However alph file expected version is not updated.

Add tutorials/samples for common use cases

Alpheus seems to hang as no info is reported

Sometime Alpheus can do long operations (e.g. hashing lots of data) and seems to be hanged (as nothing is printed) if run with standard verbosity.

Failed command vertex due to unexistend executable is user error

Now is represented as alpheus internal failure

Killing the methods execution during alpheus abort

If the user hits Ctrl+C to terminate the alpheus, the ongoing methods are still being computed.

The expected behaviour is that all of the child processes are closed together with alpheus process.

Enable versioning for alpheus files

Reduce does not work when output is dir

Created reduce command alpheus build -d ..\code\gather.R -d per_location_data\*.csv -o per_location_data_gathered/ RScript $in1 $in2 $out1

But alpheus runs it as vector:

ExecutionOutput: [SIR_IR_min_bound/per_location_data_gathered/ [stdout]]:       [1] "input pattern is C:\\repos\\github\\covid19-global-forecasting\\SIR_IR_min_bound\\per_location_data\\@Algeria.csv"
ExecutionOutput: [SIR_IR_min_bound/per_location_data_gathered/ [stdout]]:       [1] "output dir is C:\\repos\\github\\covid19-global-forecasting\\SIR_IR_min_bound\\per_location_data\\@Austria.csv"

Failed test: ItisLab.Alpheus.Tests.Vector scenarios.Scatter: Creates many files and the processes them as a vector [106ms]

See https://travis-ci.org/itislab/alpheus/builds/587396043?utm_source=github_status&utm_medium=notification

Test run in progress. X ItisLab.Alpheus.Tests.Vector scenarios.Scatter: Creates many files and the processes them as a vector [106ms]
Error Message:
Expected successful operation, but got error: SystemError
"Failed to compute the artefacts:
Newtonsoft.Json.JsonWriterException: Token PropertyName in state Property would result in an invalid JSON object. Path 'Origin.Fields[0].Outputs[0].Hash'.
at Newtonsoft.Json.JsonWriter.AutoComplete(JsonToken tokenBeingWritten)
at Newtonsoft.Json.JsonTextWriter.WritePropertyName(String name)
at [email protected](Tuple2 tupledArg) in /home/travis/build/itislab/alpheus/AlpheusCore/CustomSerializers.fs:line 24 at Microsoft.FSharp.Collections.SeqModule.Iterate[T](FSharpFunc2 action, IEnumerable1 source) at [email protected](Tuple2 tupledArg, MdMap2 value) in /home/travis/build/itislab/alpheus/AlpheusCore/CustomSerializers.fs:line 20 at ItisLab.Alpheus.CustomSerializers.ArtefactVersionConverter.WriteJson(JsonWriter writer, Object value, JsonSerializer serializer) in /home/travis/build/itislab/alpheus/AlpheusCore/CustomSerializers.fs:line 28 at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeConvertable(JsonWriter writer, JsonConverter converter, Object value, JsonContract contract, JsonContainerContract collectionContract, JsonProperty containerProperty) at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty) at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeList(JsonWriter writer, IEnumerable values, JsonArrayContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty) at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty) at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.Serialize(JsonWriter jsonWriter, Object value, Type objectType) at Newtonsoft.Json.Converters.DiscriminatedUnionConverter.WriteJson(JsonWriter writer, Object value, JsonSerializer serializer) at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeConvertable(JsonWriter writer, JsonConverter converter, Object value, JsonContract contract, JsonContainerContract collectionContract, JsonProperty containerProperty) at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty) at Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.Serialize(JsonWriter jsonWriter, Object value, Type objectType) at Newtonsoft.Json.JsonSerializer.SerializeInternal(JsonWriter jsonWriter, Object value, Type objectType) at Newtonsoft.Json.JsonConvert.SerializeObjectInternal(Object value, Type type, JsonSerializer jsonSerializer) at ItisLab.Alpheus.AlphFiles.save(AlphFile alphfile, String filepath) in /home/travis/build/itislab/alpheus/AlpheusCore/AlphFiles.fs:line 55 at [email protected](Unit unitVar0) in /home/travis/build/itislab/alpheus/AlpheusCore/DependencyGraph.fs:line 197 at ItisLab.Alpheus.DependencyGraph.ArtefactVertex.SaveAlphFile() in /home/travis/build/itislab/alpheus/AlpheusCore/DependencyGraph.fs:line 194 at [email protected](LinkToArtefact out) in /home/travis/build/itislab/alpheus/AlpheusCore/DependencyGraph.fs:line 382 at Microsoft.FSharp.Collections.SeqModule.Iterate[T](FSharpFunc2 action, IEnumerable1 source) at [email protected](Unit _arg5) in /home/travis/build/itislab/alpheus/AlpheusCore/DependencyGraph.fs:line 382 at Microsoft.FSharp.Control.AsyncPrimitives.CallThenInvokeNoHijackCheck[a,b](AsyncActivation1 ctxt, FSharpFunc2 userCode, b result1) at [email protected](AsyncActivation1 ctxt)
at Microsoft.FSharp.Control.Trampoline.Execute(FSharpFunc2 firstAction) --- End of stack trace from previous location where exception was thrown --- at Microsoft.FSharp.Control.AsyncResult1.Commit()
at Microsoft.FSharp.Control.AsyncPrimitives.RunSynchronouslyInCurrentThread[a](CancellationToken cancellationToken, FSharpAsync1 computation) at Microsoft.FSharp.Control.AsyncPrimitives.RunSynchronously[T](CancellationToken cancellationToken, FSharpAsync1 computation, FSharpOption1 timeout) at Microsoft.FSharp.Control.FSharpAsync.RunSynchronously[T](FSharpAsync1 computation, FSharpOption1 timeout, FSharpOption1 cancellationToken)
at ItisLab.Alpheus.ComputationGraph.CommandMethod.Execute(FSharpList1 inputs, FSharpOption1 _arg1) in /home/travis/build/itislab/alpheus/AlpheusCore/ComputationGraph.fs:line 59
at Angara.Execution.Runtime1.buildEvaluation(m v, FSharpList1 index, UInt64 time, State2 state, CancellationTokenSource cts, Boolean doContinue, Unit unitVar0)" Expected: True Actual: False Stack Trace: at ItisLab.Alpheus.Tests.Utils.assertResultOk[a,b](FSharpResult2 result) in /home/travis/build/itislab/alpheus/AlpheusUnitTests/TestUtils.fs:line 95
at <StartupCode$AlpheusUnitTests>.$ApiTests.Vectors.Scatter: Creates many files and the processes them as a [email protected](FSharpResult2 _arg7) in /home/travis/build/itislab/alpheus/AlpheusUnitTests/ApiTests.Vectors.fs:line 136 at Microsoft.FSharp.Control.AsyncPrimitives.CallThenInvokeNoHijackCheck[a,b](AsyncActivation1 ctxt, FSharpFunc2 userCode, b result1) at [email protected](TResult result1) at Microsoft.FSharp.Control.AsyncActivation1.OnSuccess(T result)
at [email protected](AsyncActivation1 ctxt) in /home/travis/build/itislab/alpheus/AlpheusCore/API.fs:line 382 at [email protected](AsyncActivation1 ctxt)
at [email protected](AsyncActivation1 ctxt) in /home/travis/build/itislab/alpheus/AlpheusCore/API.fs:line 377 at [email protected](AsyncActivation1 ctxt) in /home/travis/build/itislab/alpheus/AlpheusCore/DependencyGraph.fs:line 462
at <StartupCode$FSharp-Core>.$[email protected](AsyncActivation1 ctxt) at <StartupCode$FSharp-Core>[email protected](AsyncActivation1 ctxt)
at Microsoft.FSharp.Control.Trampoline.Execute(FSharpFunc`2 firstAction)
--- End of stack trace from previous location where exception was thrown ---
--- End of stack trace from previous location where exception was thrown ---
Standard Output Messages:
Creating unique test dir data/singleTimeDirs/35c1f4d5-e508-474a-a9fe-60b16c3ac0ea/
API: Outputs: [Path "1_2.txt"]
API: Dependencies: []
API: Outputs: [Path "samples/.txt"]
API: Command: "/bin/sh -c "for i in $(seq 1 3); do echo sample$i > sample$i.txt; done""
API: Building the dependency graph
API: Dependency graph is built (0 artefacts; 0 methods)
DependencyGraph: Saving alph file for artefact Path "samples/.txt"
DependencyGraph: Saved alph file for artefact Path "samples/.txt"
API: Command: "/bin/sh -c "cat $in1 > $out1; cat $in2 >> $out1""
API: Building the dependency graph
API: Dependency graph is built (2 artefacts; 2 methods)
API: Dependency graph is built (1 artefacts; 1 methods)
DependencyGraph: Saving alph file for artefact Path "1_2.txt"
API: Graph artefacts: [Artefact(samples/.txt|)]
DependencyGraph: Saved alph file for artefact Path "1_2.txt"
API: Dependency graph is built (3 artefacts; 3 methods)
API: Building the dependency graph
API: Graph artefacts: [Artefact(1.txt|); Artefact(2.txt|); Artefact(1_2.txt|)]
API: Dependency graph is built (1 artefacts; 1 methods)
API: Running computations
API: Dependencies: [Path "1_2.txt"; Path "3.txt"]
API: Outputs: [Path "1_2_3.txt"]
API: Command: "/bin/sh -c "cat $in1 > $out1; cat $in2 >> $out1""
API: Building the dependency graph
API: Dependency graph is built (4 artefacts; 4 methods)
DependencyGraph: Saving alph file for artefact Path "1_2_3.txt"
DependencyGraph: Saved alph file for artefact Path "1_2_3.txt"
API: Dependency graph is built (5 artefacts; 5 methods)
API: Graph artefacts: [Artefact(1.txt|); Artefact(2.txt|); Artefact(3.txt|); Artefact(1_2.txt|);
Artefact(1_2_3.txt|)]
API: Dependencies: [Path "dir1/"]
Execution: samples/.txt[]: Started
API: Outputs: [Path "dir1_copy/"]
Execution: samples/.txt: Needs recomputation as disk version of the output does not match expected version
API: Command: "/bin/sh -c "rm -Rv $out1 ; cp -Rv $in1 $out1""
API: Building the dependency graph
Execution: [ samples/.txt]: Running "/bin/sh -c "for i in $(seq 1 3); do echo sample$i > sample$i.txt; done"" in "/home/travis/build/itislab/alpheus/AlpheusUnitTests/bin/Debug/netcoreapp2.1/data/singleTimeDirs/971b3d66-5ad8-4c64-a04a-e191d2726470/data/singleTimeDirs/35c1f4d5-e508-474a-a9fe-60b16c3ac0ea/samples/"
API: Dependency graph is built (1 artefacts; 1 methods)
DependencyGraph: Saving alph file for artefact Path "dir1_copy/"
DependencyGraph: Saved alph file for artefact Path "dir1_copy/"
API: Dependency graph is built (2 artefacts; 2 methods)
API: Graph artefacts: [Artefact(dir1/|); Artefact(dir1_copy/|)]
Execution: [ samples/.txt]: Started subprocess (PID=5025)
Execution: [ samples/.txt]: Subprocess (PID=5025) exited with exit code 0
Execution: samples/.txt[]: Method succeeded
Execution: samples/.txt[]: Program succeeded. Calculating hashes of the outputs...
DependencyGraph: Method "samples/.txt"[[]] succeeded
DependencyGraph: Calculating hash of /home/travis/build/itislab/alpheus/AlpheusUnitTests/bin/Debug/netcoreapp2.1/data/singleTimeDirs/971b3d66-5ad8-4c64-a04a-e191d2726470/data/singleTimeDirs/35c1f4d5-e508-474a-a9fe-60b16c3ac0ea/samples/sample2.txt...
DependencyGraph: Calculating hash of /home/travis/build/itislab/alpheus/AlpheusUnitTests/bin/Debug/netcoreapp2.1/data/singleTimeDirs/971b3d66-5ad8-4c64-a04a-e191d2726470/data/singleTimeDirs/35c1f4d5-e508-474a-a9fe-60b16c3ac0ea/samples/sample3.txt...
Hash: Artefact /home/travis/build/itislab/alpheus/AlpheusUnitTests/bin/Debug/netcoreapp2.1/data/singleTimeDirs/971b3d66-5ad8-4c64-a04a-e191d2726470/data/singleTimeDirs/35c1f4d5-e508-474a-a9fe-60b16c3ac0ea/samples/sample3.txt - acquiring data hash...
Hash: Artefact /home/travis/build/itislab/alpheus/AlpheusUnitTests/bin/Debug/netcoreapp2.1/data/singleTimeDirs/971b3d66-5ad8-4c64-a04a-e191d2726470/data/singleTimeDirs/35c1f4d5-e508-474a-a9fe-60b16c3ac0ea/samples/sample2.txt - acquiring data hash...
DependencyGraph: Calculating hash of /home/travis/build/itislab/alpheus/AlpheusUnitTests/bin/Debug/netcoreapp2.1/data/singleTimeDirs/971b3d66-5ad8-4c64-a04a-e191d2726470/data/singleTimeDirs/35c1f4d5-e508-474a-a9fe-60b16c3ac0ea/samples/sample1.txt...
Hash: Artefact /home/travis/build/itislab/alpheus/AlpheusUnitTests/bin/Debug/netcoreapp2.1/data/singleTimeDirs/971b3d66-5ad8-4c64-a04a-e191d2726470/data/singleTimeDirs/35c1f4d5-e508-474a-a9fe-60b16c3ac0ea/samples/sample1.txt - acquiring data hash...
Hash: Artefact /home/travis/build/itislab/alpheus/AlpheusUnitTests/bin/Debug/netcoreapp2.1/data/singleTimeDirs/971b3d66-5ad8-4c64-a04a-e191d2726470/data/singleTimeDirs/35c1f4d5-e508-474a-a9fe-60b16c3ac0ea/samples/sample1.txt hash is calculated (didn't find precomputed .hash file)
Successfully deleted unique test dir data/singleTimeDirs/35c1f4d5-e508-474a-a9fe-60b16c3ac0ea/

itislab / alpheus Goto Github PK

alpheus's Issues

Recommend Projects

Recommend Topics

Recommend Org