Comments (6)
I understood. Don't use hack. In ansible install I used a configuration file test that doesn't have a new configuration parameters. So add this and it will work.
from spamscope.
Ah I see that now. Added and works now. Great thank you!
from spamscope.
I can't help you with only that log. Are there any Traceback
?
from spamscope.
Those are the errors appearing in the Storm UI. The test emails all failed and are in the failed folder. Logs in /var/log/spamscope are empty(files appear but they are empty).
Is there another place I can find the traceback being thrown?
I am going to reinstall to make sure I didn’t miss a step.
BTW, awesome tool once I get it working.... Would love to assist if need be.
from spamscope.
I put a hack in to make it work and Im sure I probably missed something. I edited the tokenizer.py file and changed self.persistent_path to a static path /home/myusername. After making this edit, it started to work. Is there a permanent solution to this?
Found it:
2018-10-10 09:16:48.454 o.a.s.d.executor Thread-22-urls-executor[10 10] [INFO] Prepared bolt urls:(10)
2018-10-10 09:16:48.456 o.a.s.d.executor Thread-24-json_maker-executor[4 4] [INFO] Prepared bolt json_maker:(4)
2018-10-10 09:16:48.459 phishing Thread-38 [INFO] /opt/spamscope/venv/local/lib/python2.7/site-packages/astropy/config/configuration.py:541: ConfigurationMissingWarning: Configuration defaults will be used due to OSError:Could not find unix home directory to search for astropy config dir on None
warn(ConfigurationMissingWarning(msg))
Traceback (most recent call last):
File "/opt/spamscope/venv//bin/streamparse_run", line 11, in <module>
sys.exit(main())
File "/opt/spamscope/venv/local/lib/python2.7/site-packages/streamparse/run.py", line 45, in main
cls(serializer=args.serializer).run()
File "/opt/spamscope/venv/local/lib/python2.7/site-packages/pystorm/component.py", line 485, in run
self.initialize(storm_conf, context)
File "/var/lib/storm/supervisor/stormdist/spamscope_debug-1-1539177259/resources/bolts/tokenizer.py", line 73, in initialize
self.load_filters()
File "/var/lib/storm/supervisor/stormdist/spamscope_debug-1-1539177259/resources/bolts/tokenizer.py", line 83, in load_filters
path = self.get_persistent_path(i)
File "/var/lib/storm/supervisor/stormdist/spamscope_debug-1-1539177259/resources/bolts/tokenizer.py", line 67, in get_persistent_path
self.persistent_path, "{}.dump".format(filter_name))
File "/var/lib/storm/supervisor/stormdist/spamscope_debug-1-1539177259/resources/bolts/tokenizer.py", line 63, in __getattr__
return self.conf[name]
KeyError: 'persistent_path'
2018-10-10 09:16:48.479 o.a.s.t.ShellBolt Thread-33 [ERROR] Halting process: ShellBolt died. Command: [/opt/spamscope/venv//bin/streamparse_run, -s json bolts.tokenizer.Tokenizer], ProcessInfo pid:26136, name:tokenizer exitCode:-1, errorString:
java.lang.RuntimeException: org.apache.storm.multilang.NoOutputException: Pipe to subprocess seems to be broken! No output read.
Serializer Exception:
at org.apache.storm.utils.ShellProcess.readShellMsg(ShellProcess.java:127) ~[storm-core-1.2.1.jar:1.2.1]
at org.apache.storm.task.ShellBolt$BoltReaderRunnable.run(ShellBolt.java:330) [storm-core-1.2.1.jar:1.2.1]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]
2018-10-10 09:16:48.495 o.a.s.d.executor Thread-33 [ERROR]
java.lang.RuntimeException: org.apache.storm.multilang.NoOutputException: Pipe to subprocess seems to be broken! No output read.
Serializer Exception:
at org.apache.storm.utils.ShellProcess.readShellMsg(ShellProcess.java:127) ~[storm-core-1.2.1.jar:1.2.1]
at org.apache.storm.task.ShellBolt$BoltReaderRunnable.run(ShellBolt.java:330) [storm-core-1.2.1.jar:1.2.1]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]
2018-10-10 09:16:48.526 o.a.s.d.worker Thread-27 [INFO] Shutting down worker spamscope_debug-1-1539177259 5f98d79b-abe4-4fed-b5e8-33425d5d913d 6701
2018-10-10 09:16:48.530 o.a.s.d.worker Thread-27 [INFO] Terminating messaging context
2018-10-10 09:16:48.531 o.a.s.d.worker Thread-27 [INFO] Shutting down executors
2018-10-10 09:16:48.543 o.a.s.d.executor Thread-27 [INFO] Shutting down executor raw_mail:[8 8]
2018-10-10 09:16:48.550 o.a.s.util Thread-4-raw_mail-executor[8 8] [INFO] Async loop interrupted!
2018-10-10 09:16:48.550 o.a.s.util Thread-3-disruptor-executor[8 8]-send-queue [INFO] Async loop interrupted!
2018-10-10 09:16:48.556 o.a.s.t.ShellBolt Thread-32 [ERROR] Halting process: ShellBolt died. Command: [/opt/spamscope/venv//bin/streamparse_run, -s json bolts.raw_mail.RawMail], ProcessInfo pid:26128, name:raw_mail exitCode:-1, errorString:
java.lang.RuntimeException: org.apache.storm.multilang.NoOutputException: Pipe to subprocess seems to be broken! No output read.
Serializer Exception:
from spamscope.
I fixed the configuration file: 33599ea
from spamscope.
Related Issues (20)
- Migration to streamparse 3.x HOT 1
- New picking up emails and not recognized index in kibana HOT 2
- Manage Rejecting mapping update to in Elasticsearch bolt
- Configuration defaults will be used due to OSError HOT 1
- Sender Ip is always NULL HOT 2
- Add more lookups (other than Virustotal and Shodan) HOT 1
- Unable to convert Float HOT 3
- SpamAssassin returns empty dictionary HOT 6
- Unsure about data input
- Serializer Exception & Pipe Broken HOT 1
- Elasticsearch output bolt HOT 2
- Exception in phishing analysis for mail with mutiple subject headers HOT 2
- I noticed my issue was deleted, is this library not supported anymore? HOT 3
- Split actual output in: JSON mails and JSON attachments HOT 1
- Change the way to manage the inputs from more than one bolt HOT 1
- Reload configuration only if it changes HOT 2
- Integration with oletools HOT 1
- Integration with Shodan HOT 2
- Consider swapping out tika-app with tika-python HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from spamscope.