I tryed to configure a workflow with 2 actions, one after the other.
The actions are

• get active hosts from nmap app
• json select from utilities app

I had many difficulties to understand what is returned from the first action, so I modified the json select action to print what is receiving as an input in order to understand.

Turns out that the actual received result is different from the result shown in the gui.
From the gui I see the result

But by printing the result in json select I can see that only the value of the json[result][result]

Is there a reason behind this difference? Is there a way to know what is returned from an action different from reading the code?

Many thanks

The back-end currently supports adding multiple error-handling steps. This capability is not supported by the front end.

Currently the back-end supports adding breakpoints to workflow steps allowing a user to step through a workflow. This capability is not currently supported by the UI

A session is currently required to connect to the backend. It would be better and more RESTful to make the API key sessionless

During the transition from playbooks to workflows, the file extension has not be renamed. This is confusing.

Triggers would be more powerful if they were reimagined as a generalized event-driven step. So a workflow would await a call from our REST API. The step wouldn't have an app or an action, but instead of a list of Flags. This would allow triggers to exist in the middle of workflows. The existing functionality would be replicated by either putting a trigger step at the beginning of a workflow or connecting a new endpoint to directly execute a workflow.

I'm not sure if this is on purpose but PyCharm is warning that the height and width values are being overwritten.

/* server/templates/pages/playbook/index.html */

#parametersBar{
    height: 651px; /* == 700 minus 49 (where 49 is the height of the tab) */
    width: 450px;
    display: inline-block;
    vertical-align: top;
    height: 700px;  /* this overwrites the height above */
    width: 400px;  /* this overwrites the width above */
    overflow-y:auto;
    background-color:#f5f5f5;
}

Currently all flags on a nextstep must be evaluated as true for a step to proceed to that step. It would be beneficial to allow "or" logic, meaning that one set of flags or another set of flags must be satisfied to progress to another step. A workaround is to create a new flag which includes this logic, but that is not a great solution. A good implementation could be challenging.

There are multiple collisions in the db_path, case_db_path, walkoff.config location when running the test suite on system that has also run the startServer.py.

The application should have at least three distinct environments – production, development, test, and often staging; and quite often the application will behave differently in each environment.

If you run startServer, and create any content such as triggers, workbooks etc the test counts will likely fail.

Test that measure the counts in the database fail such as:

test_add_and_display_and_remove_trigger (tests.test_triggers.TestTriggers) ... FAIL

======================================================================
FAIL: test_add_and_display_and_remove_trigger (tests.test_triggers.TestTriggers)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/WALKOFF/tests/test_triggers.py", line 48, in test_add_and_display_and_remove_trigger
    self.assertEqual(len(triggers), 1)
AssertionError: 2 != 1

Currently the widgets do not display properly on the dashboard

Currently the controller introspects the apps module to find the correct app to construct at run time. We could reduce this by making the apps.App class inherit a metaclass which has a registry. A similar approach has been used in flags and filters, and now that the apps.App class does not inherit the SQLAlchemy metaclass, we don't need to deal with the issue of metaclass multiple inheritance. This new architecture would further reduce dependencies between the apps and the controller modules.

Currently all the flags and filters are global and contained in the core module. We should allow apps to have their own custom flags and filters to be used for app-specific actions and data transformations

Currently arguments can only be a string or int. Adding more primitive types would be trivial, but adding more complex data types and better validators would get increasingly difficult. Using a format similar to Swagger's OpenAPI specification used throughout the server, we could provide more advanced validation tools. We may even be able to make custom validators which would allow raw Python objects to be used as arguments. This would greatly enhance flexibility and usability of the apps and might possibly open the door to app documentation generation similar to Swagger UI.

Currently we use a threadpool to parallelize workflow execution. The gevent monkey-patching interferes with the multiprocessing library. We could use something like Celery, which integrates easily into Flask, to due better multiprocessing

Currently, app schemas cannot use JSON references of the form "#/path/to/schema". These references must be dereferenced upon schema validation check on initialization and stored. An incorrect implementation is currently in core/schemas/dereference.py, but it is not incorporated into the validation. Additionally, it is implemented as a recursively; it should probably use generators to reduce memory overhead.

The ability to have app action to act as a listener which triggers another step could be a very useful feature.

Currently, the user does not have a way of checking the status of a workflow. They can pause a workflow and resume a workflow and the result of the workflow is displayed on the UI after it has completed, but while it is executing there is no way to check its status.

This could be fixed by generating a UID for each workflow being executed. This UID would be used to manipulate a workflow and check on its status. Some difficulty would come when some when someone would want to check on the status of a workflow that was completed long ago. We might not be able to keep the workflow status in memory as more workflows are executed.

Currently the ability to use multiple error-handling steps is difficult and would mostly rely on doing regular expressions on the error message.

Current Jinja templating is used to use the result of the previous step in the next step. This behavior should be enabled by default.

This task should probably be done after we transition to using OpenAPI for app actions.

the core.controller object is becoming bloated and rigid. I propose that we break it up as follows (pseudocode):

class WorkflowExecutor:
	self.executor = MultiprocessesExecutor()
	self.playbook_store = PlaybookStore()
	self.scheduler = scheduler()

class MultiprocessedExecutor():
	def execute_workflow(Workflow)
	def pause_workflow(workflow_execution_uid, workflow_name)
	def resume_workflow(workflow_execution_uid, workflow_name)
	def _event_callback
		# maybe all execution elements could call a single blinker signal. MultiprocessedExecutor connects that signal. No need for injecting callbacks deep into workflow structure
	def shutdown()

class PlaybookStore():
	self.playbooks = {playbook_name: Playbook}
	self.playbook_loader = JsonPlaybookLoader()

class JsonPlaybookLoader:
	# These may all be static for some loaders. Keep them in a class. Essentially allow module to be used as a field in PlaybookStore
	def load_workflow(resource_location, workflow_name, playbook_name)
	def load_playbook(resource_location)
	def load_all(resource_collection_location)

class Playbook():
	self.name
	self.workflows = {uid: Workflow}
	def as_json()

Link shows https when there is no SSL context, and the service is actually http

[2017-06-26 10:04:04,156]  startserver::INFO - Listening on host https://127.0.0.1:5000

First Issue, the setup.py , initially Flask.ext.security and flask.ext.sqlalchemy was installed but the following were needed and were not installed so we installed it manually

                    'Flask_Principal >= 0.4.0',
                    'Flask_SQLAlchemy >= 2.1',
                    'Flask_Security >= 1.7.5',

When I try to run main.py , we are having this issue with mainAPI.py

[root@bn-r7-walkoff WALKOFF]# python main.py
/root/WALKOFF/apps/resources
Traceback (most recent call last):
  File "main.py", line 1, in <module>
    import api.mainAPI as webService
  File "/root/WALKOFF/api/mainAPI.py", line 153, in <module>
    security = Security(app, user_datastore)
  File "/usr/lib/python2.7/site-packages/Flask_Security-1.7.5-py2.7.egg/flask_security/core.py", line 401, in __init__
    self._state = self.init_app(app, datastore, **kwargs)
  File "/usr/lib/python2.7/site-packages/Flask_Security-1.7.5-py2.7.egg/flask_security/core.py", line 435, in init_app
    anonymous_user=anonymous_user)
  File "/usr/lib/python2.7/site-packages/Flask_Security-1.7.5-py2.7.egg/flask_security/core.py", line 268, in _get_state
    login_manager=_get_login_manager(app, anonymous_user),
  File "/usr/lib/python2.7/site-packages/Flask_Security-1.7.5-py2.7.egg/flask_security/core.py", line 226, in _get_login_manager
    lm.token_loader(_token_loader)
AttributeError: 'LoginManager' object has no attribute 'token_loader'

It would be useful to have the ability to execute steps in parallel. This might be a very difficult feature to implement on the backend, but maybe each workflow process pool could have its own smaller threadpool. Furthermore graphically depicting this on the UI could be challenging.

  super(LoginForm, self).__init__(*args, **kwargs)
./home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/flask_security/forms.py:213: FlaskWTFDeprecationWarning: "flask_wtf.Form" has been renamed to "FlaskForm" and will be removed in 1.0.

The deprecation warning lead to an update in flask, flask_wtf, Flask_SQLAlchemy, and flask_security

Flask >= 0.11.0
Flask_WTF >= 0.13.1, <1 (top side pinning)
Flask_Security == 3.0.0
Flask_SQLAlchemy >= 2.1, <3 (top side pinning)

The tests module is becoming bloated and needs to be split into submodules.

Currently the back-end supports a concept of risk associated with an executing workflow. This is not shown or supported by the UI.

Goal
The app should be able to securely execute a PowerShell command and script on a remote host.

Functions
executeCommand
The command should be a string argument passed in through the workflow

executeScript
The script should be a filename passed in through the workflow

A PEP8 review should be done on the code base on PR and .travis build automatically.

There should be a tool integration like:

• codeclimate pep8
• coverall.io
• codecov.io
• https://github.com/OrkoHunter/pep8speaks

pycodestyle --statistics -qq ./
10      E101 indentation contains mixed spaces and tabs
194     E111 indentation is not a multiple of four
8       E114 indentation is not a multiple of four (comment)
20      E127 continuation line over-indented for visual indent
90      E128 continuation line under-indented for visual indent
1       E129 visually indented line with same indent as next logical line
21      E201 whitespace after '('
31      E202 whitespace before ')'
8       E203 whitespace before ':'
32      E221 multiple spaces before operator
11      E222 multiple spaces after operator
5       E231 missing whitespace after ','
17      E251 unexpected spaces around keyword / parameter equals
36      E261 at least two spaces before inline comment
1       E262 inline comment should start with '# '
6       E265 block comment should start with '# '
2       E266 too many leading '#' for block comment
1       E301 expected 1 blank line, found 0
52      E302 expected 2 blank lines, found 1
6       E303 too many blank lines (3)
13      E305 expected 2 blank lines after class or function definition, found 1
3       E306 expected 1 blank line before a nested definition, found 0
4       E401 multiple imports on one line
1       E402 module level import not at top of file
2997    E501 line too long (146 > 79 characters)
8       E703 statement ends with a semicolon
1       E721 do not compare types, use 'isinstance()'
12      E722 do not use bare except'
133     W191 indentation contains tabs
85      W291 trailing whitespace
26      W292 no newline at end of file
263     W293 blank line contains whitespace
23      W391 blank line at end of file
7       W602 deprecated form of raising exception

Currently some tests still rely on the apps module and the data module. This dependence on user-data should be removed and mocked in the tests module. This could involve making a testing configuration file and loading that as well as creating two base classes for test cases, one which is used for non-server tests and one which is used for server tests.

When an app fails or a step's input validation fails, the logger should record the reason. It currently misses some of them.

Currently the back-end supports some limited metrics of workflow and app actions. This information is not visible on the UI.

Currently the apps are bound to this repo, but they should change rapidly. Additionally, they are tested in Travis along with the core. The apps should be in their own repo which is a subrepo of this repo. To test the, the apps.App class can be mocked so the apps can be tested separately from this repo. The apps repo could be connected to this repo using a subtree or a submodule

LC_COLLATE='C' operating systems sort files in a different order, causing the unittest to fail.

os.listdir(path) should all be wrapped by sorted() for ordered dictionaries.

Reference PR #125 - core/helpers.py

Currently a steps name is used as a unique identifier, so it must be generated on the UI. It would be much better to allow a user to name the steps and use a proper unique identifier in the backend. This UID would be stored in the playbook XML. This would require modifying the steps object.

Currently adding this feature would break the case management system on the UI because it relies on the user entering an element's name by hand. Until the user is no longer required to do this, this feature will not work.

Furthermore, if we make all execution elements have unique id's then the concept of "ancestry" can be removed and the subscription's tree data structure can also be replaced by a list of UIDs. This would greatly reduce complexity and should increase performance

The name on the step nodes on the workflow editor is currently the name of the method. It would be cleaner and easier to read if the method had a string to use a display name in its metadata.

Currently the back-end supports pausing and resuming workflows. The capability is not supported in the UI.

Currently the back-end supports loading and executing a workflow which contains a step which has another workflow as its child step. However, this support is not well-developed and there is no method to control it through the server.

Currently the architecture of the roles is somewhat limited. It would be nice to do something like read/write/execute permissions on different endpoints and different apps