logstash-plugins / logstash-filter-cipher Goto Github PK

License: Apache License 2.0

Ruby 100.00%

logstash-filter-cipher's Introduction

Logstash Plugin

This is a plugin for Logstash.

It is fully free and fully open source. The license is Apache 2.0, meaning you are pretty much free to use it however you want in whatever way.

Documentation

Logstash provides infrastructure to automatically generate documentation for this plugin. We use the asciidoc format to write documentation so any comments in the source code will be first converted into asciidoc and then into html. All plugin documentation are placed under one central location.

For formatting code or config example, you can use the asciidoc [source,ruby] directive
For more asciidoc formatting tips, see the excellent reference here https://github.com/elastic/docs#asciidoc-guide

Need Help?

Need help? Try #logstash on freenode IRC or the https://discuss.elastic.co/c/logstash discussion forum.

Developing

1. Plugin Developement and Testing

Code

To get started, you'll need JRuby with the Bundler gem installed.
Create a new plugin or clone and existing from the GitHub logstash-plugins organization. We also provide example plugins.
Install dependencies

bundle install

Test

Update your dependencies

bundle install

Run tests

bundle exec rspec

2. Running your unpublished Plugin in Logstash

2.1 Run in a local Logstash clone

Edit Logstash Gemfile and add the local plugin path, for example:

gem "logstash-filter-awesome", :path => "/your/local/logstash-filter-awesome"

Install plugin

# Logstash 2.3 and higher
bin/logstash-plugin install --no-verify

# Prior to Logstash 2.3
bin/plugin install --no-verify

Run Logstash with your plugin

bin/logstash -e 'filter {awesome {}}'

At this point any modifications to the plugin code will be applied to this local Logstash setup. After modifying the plugin, simply rerun Logstash.

2.2 Run in an installed Logstash

You can use the same 2.1 method to run your plugin in an installed Logstash by editing its Gemfile and pointing the :path to your local plugin development directory or you can build the gem and install it using:

Build your plugin gem

gem build logstash-filter-awesome.gemspec

Install the plugin from the Logstash home

# Logstash 2.3 and higher
bin/logstash-plugin install --no-verify

# Prior to Logstash 2.3
bin/plugin install --no-verify

Start Logstash and proceed to test the plugin

Contributing

All contributions are welcome: ideas, patches, documentation, bug reports, complaints, and even something you drew up on a napkin.

Programming is not a required skill. Whatever you've seen about open source and maintainers or community members saying "send patches or die" - you will not see that here.

It is more important to the community that you are able to contribute.

For more information about contributing, see the CONTRIBUTING file.

logstash-filter-cipher's People

Contributors

Stargazers

Watchers

logstash-filter-cipher's Issues

After logstash restart, it takes about 10 minutes to encrypt the first message

Problems: As title. Cipher is as follow:
cipher {
mode => "encrypt"
algorithm => "des-cbc"
iv_random_length => 8
key => "@CIPHER_KEY_TOKEN@"
source => "data"
target => "data"
#enable_metric => false
}

Use case with KMS

Tell us about the issue

Lack of documentation around kms

Description:
I have wired up rds logs --> kinesis via database activity stream, which is then streamed to elk via a "logstash pipeline". One tiny problem here is that the rds logs are all just a giant blob of mess getting spammed to elk.
There seems to be this plugin (looks to be deprecated...) that deals with kms. Are there any documentation within this repo around integration specifically with kms?

cipher {
    algorithm => "aes-128-cbc"
    cipher_padding => 1

    iv_random_length => 16
    key => "WHATS_IN_HERE?"   <----
    key_size => 16

    mode => "decrypt"
    source => "message"
    target => "message"
    base64 => true
}

Anything else?

Unable to use authenticated encryption methods

Unfortunately, Ruby's OpenSSL plugin as shipped with v0.1.4 doesn't support any authenticated encryption algorithms, so it's impossible to tell if anyone tampered with my data after encryption. This is the output from Ruby:

> require 'openssl'
> puts OpenSSL::Cipher.ciphers
AES-128
aes-128
AES-128-CBC
aes-128-cbc
AES-128-CFB
aes-128-cfb
AES-128-CFB1
aes-128-cfb1
AES-128-CFB8
aes-128-cfb8
AES-128-ECB
aes-128-ecb
AES-128-OFB
aes-128-ofb
AES-192
aes-192
AES-192-CBC
aes-192-cbc
AES-192-CFB
aes-192-cfb
AES-192-CFB1
aes-192-cfb1
AES-192-CFB8
aes-192-cfb8                                                                                                
AES-192-ECB
aes-192-ecb
AES-192-OFB
aes-192-ofb
AES-256
aes-256
AES-256-CBC
aes-256-cbc
AES-256-CFB
aes-256-cfb
AES-256-CFB1
aes-256-cfb1
AES-256-CFB8
aes-256-cfb8
AES-256-ECB
aes-256-ecb
AES-256-OFB
aes-256-ofb
BF
bf
BF-CBC
bf-cbc
BF-CFB
bf-cfb
BF-CFB1
bf-cfb1
BF-CFB8
bf-cfb8
BF-ECB
bf-ecb
BF-OFB
bf-ofb
DES
des
DES-CBC
des-cbc
DES-CFB
des-cfb
DES-CFB1
des-cfb1
DES-CFB8
des-cfb8
DES-ECB
des-ecb
DES-OFB
des-ofb
DES-EDE
des-ede
DES-EDE-CBC
des-ede-cbc
DES-EDE-CFB
des-ede3-cfb
DES-EDE3-CFB1
des-ede3-cfb1
DES-EDE3-CFB8
des-ede3-cfb8
DES-EDE3-ECB
des-ede3-ecb
DES-EDE3-OFB
des-ede3-ofb
RC2
rc2
RC2-CBC
rc2-cbc
RC2-CFB
rc2-cfb
RC2-CFB1
rc2-cfb1
RC2-CFB8
rc2-cfb8
RC2-ECB
rc2-ecb
RC2-OFB
rc2-ofb
CAST5
cast5
CAST5-CBC
cast5-cbc
CAST5-CFB
cast5-cfb
CAST5-CFB1
cast5-cfb1
CAST5-CFB8
cast5-cfb8
CAST5-ECB
cast5-ecb
CAST5-OFB
cast5-ofb
BLOWFISH
blowfish
RC2-40-CBC
rc2-40-cbc
RC2-64-CBC
rc2-64-cbc
RC4
rc4
RC4-40
rc4-40
CAST
cast
CAST-CBC
cast-cbc

This is only a limited subset of what OpenSSL actually supports underneath the hood. It seems that the Ruby OpenSSL library isn't too recent. By upgrading the Ruby OpenSSL binding, we'd also be able to use authenticated ciphers like AES-{128,192.256}-GCM. On Ubuntu 14.04 if I just run the following, here's what I see that OpenSSL natively supports:

$ openssl ciphers | tr ":" "\n"
ECDHE-RSA-AES256-GCM-SHA384
ECDHE-ECDSA-AES256-GCM-SHA384
ECDHE-RSA-AES256-SHA384
ECDHE-ECDSA-AES256-SHA384
ECDHE-RSA-AES256-SHA
ECDHE-ECDSA-AES256-SHA
SRP-DSS-AES-256-CBC-SHA
SRP-RSA-AES-256-CBC-SHA
DHE-DSS-AES256-GCM-SHA384
DHE-RSA-AES256-GCM-SHA384
DHE-RSA-AES256-SHA256
DHE-DSS-AES256-SHA256
DHE-RSA-AES256-SHA
DHE-DSS-AES256-SHA
DHE-RSA-CAMELLIA256-SHA
DHE-DSS-CAMELLIA256-SHA
ECDH-RSA-AES256-GCM-SHA384
ECDH-ECDSA-AES256-GCM-SHA384
ECDH-RSA-AES256-SHA384
ECDH-ECDSA-AES256-SHA384
ECDH-RSA-AES256-SHA
ECDH-ECDSA-AES256-SHA
AES256-GCM-SHA384
AES256-SHA256
AES256-SHA
CAMELLIA256-SHA
PSK-AES256-CBC-SHA
ECDHE-RSA-DES-CBC3-SHA
ECDHE-ECDSA-DES-CBC3-SHA
SRP-DSS-3DES-EDE-CBC-SHA
SRP-RSA-3DES-EDE-CBC-SHA
EDH-RSA-DES-CBC3-SHA
EDH-DSS-DES-CBC3-SHA
ECDH-RSA-DES-CBC3-SHA
ECDH-ECDSA-DES-CBC3-SHA
DES-CBC3-SHA
PSK-3DES-EDE-CBC-SHA
ECDHE-RSA-AES128-GCM-SHA256
ECDHE-ECDSA-AES128-GCM-SHA256
ECDHE-RSA-AES128-SHA256
ECDHE-ECDSA-AES128-SHA256
ECDHE-RSA-AES128-SHA
ECDHE-ECDSA-AES128-SHA
SRP-DSS-AES-128-CBC-SHA
SRP-RSA-AES-128-CBC-SHA
DHE-DSS-AES128-GCM-SHA256
DHE-RSA-AES128-GCM-SHA256
DHE-RSA-AES128-SHA256
DHE-DSS-AES128-SHA256
DHE-RSA-AES128-SHA
DHE-DSS-AES128-SHA
DHE-RSA-SEED-SHA
DHE-DSS-SEED-SHA
DHE-RSA-CAMELLIA128-SHA
DHE-DSS-CAMELLIA128-SHA
ECDH-RSA-AES128-GCM-SHA256
ECDH-ECDSA-AES128-GCM-SHA256
ECDH-RSA-AES128-SHA256
ECDH-ECDSA-AES128-SHA256
ECDH-RSA-AES128-SHA
ECDH-ECDSA-AES128-SHA
AES128-GCM-SHA256
AES128-SHA256
AES128-SHA
SEED-SHA
CAMELLIA128-SHA
PSK-AES128-CBC-SHA
ECDHE-RSA-RC4-SHA
ECDHE-ECDSA-RC4-SHA
ECDH-RSA-RC4-SHA
ECDH-ECDSA-RC4-SHA
RC4-SHA
RC4-MD5
PSK-RC4-SHA
EDH-RSA-DES-CBC-SHA
EDH-DSS-DES-CBC-SHA
DES-CBC-SHA
EXP-EDH-RSA-DES-CBC-SHA
EXP-EDH-DSS-DES-CBC-SHA
EXP-DES-CBC-SHA
EXP-RC2-CBC-MD5
EXP-RC4-MD5

AES-256-GCM is listed as being supported, but it's unavailable within the the cipher filter, possibly due to an old Ruby OpenSSL bindings library.

Trouble using cipher with AWS SQS input plugin

I'm using logstash 2.2.0 to read medium volume events from AWS SQS (running 64 input threads). We recently added the cipher plugin to address the fact that SQS offers no confirmed encryption at-rest capabilities. We had some issues with logstash crashing with stack traces like so:

NoMethodError: undefined method `padding=' for nil:NilClass
and
OpenSSL::Cipher::CipherError: key not specified

Turning on debugging I saw the following messages from the cipher plugin:

max_cipher_reuse[10] reached, total_cipher_uses = 11
max_cipher_reuse[10] reached, total_cipher_uses = 12

The only possible interpretation I could come to from this was that the cipher object was being called from multiple threads, despite the fact that this shouldn't be possible given the pipeline worker architecture per https://www.elastic.co/guide/en/logstash/2.2/pipeline.html

As a quick patch to get us up and running, I put a mutex around basically the entire code that uses the OpenSSL cipher object. I realize this probably isn't the right solution but I wanted to get this documented.

I'll create a pull request with my code for reference.

logstash 1.5.4 cipher doesn't work

Hi,
I use logstash 1.5.4 and I have a problem with cipher plugin about decrypt mode.
Everytime the filter doesn't work and setting in verbose mode the agent shows the next message:

I tried different messages as input and also just one word:
I tried encrypt and decrypt in two cases, in the same agent o in different agent.
In the same agent the filter code is:

filter {
cipher {
algorithm => "aes-256-cbc"
key => "12345678901234567890123456789012"
mode => "encrypt"
source => "message"
target => "message_crypted"
add_field => { "crypted" => "TRUE"}
base64 => true
}

cipher {    
       algorithm => "aes-256-cbc"
       key => "12345678901234567890123456789012"    
       mode => "decrypt"
       source => "message_crypted"
       target => "message_uncrypted"
       add_field => { "encrypted" => "TRUE"}
       base64 => true
    }

}

I tried different combination but the result is always the same and I doesn'tunderstans if the problem is in the encrypt or decrypt:

with encryption I obtain this result:

original message: <13>root: [22120]: LENGTH: "323" SESSIONID:[8] "29180120" ENTRYID:[1] "1" STATEMENT:[1] "1" USERID:[8] "ECARLINI" USERHOST:[16] "INTERNO\ECARLINI" TERMINAL:[8] "ECARLINI" ACTION:[3] "100" RETURNCODE:[4] "1017" COMMENT$TEXT:[98] "Authenticated by: DATABASE; Client address: (ADDRESS=(PROTOCOL=tcp)(HOST=10.2.131.26)(PORT=50340))" OS$USERID:[8] "ecarlini"

crypted on console: \x190^G\x85\x16}\xA9#\xB8\xEC\xA5Z\xBF7\xF29\xC4k%H\xBA\x01RU\xA8\x8A-{\x97\x19\x80\xD8\xCB\x9C\xA1\e|\xC7\x16#a\x8A\xC4\r\xF4$1Ku\x83\xD5\x98N\x8C\xC7eS\xE0\xC9\xF0\xF1\xE7gW\x0E\xEFU\xFC\r\xE9\xFA\x90&\xAF\x04Z\x88]!\xBB\xAD\xD8\xB4\xE9\xE2\x9Cw\x8BC\x06\x97\x00\x9B\xC8\xC5\x87\x8C\xD7\xFD$\xBEn\xB5(\xE6\xA6\x9B\xA0DX\xC8\xBD\x8F\xCB\x8F\xBF\x8EV\xCAF5\veCtc\x06\xC7.\x13\xB7Zv\x06\xF2\xED\xB8\xC1M:\x19]t\x93@\x03\x98\xDA4\t\x95\xF7\xCEh0\x0E\xF1'\r@\x0E,L\x83v\xA4aF)\xE8\xAA\xE6\x93\xB6\xF63\x16\x8C\xAD\fg\xE34\xA9\t^\xAF\xA7\x98\x16\xCE\x1F\x9B\xC91\xAB\x87\x9E\xE1\xB8^\xF5O\xA9/\xA5\x91\x98\xE4ZZ\xFF\x8E\x80~;\xB3\x8E\xBE\xA8\xF8\xDE\xBAj\xE3\x15~\xD1\x87\xC1\xC6\xC0\v]#F\x161\xF3[\x16c\xFDYn\xDD\x8E\x95\xEE\xC3\xFB\xA6\xA9\xECu

crypted on elasticsearch:
0^G��}©#¸ì¥Z¿7ò9Äk%Hº�RU¨�-{��ØË�¡�|Ç�#a�Ä
ô$1Ku�Õ�N�ÇeSàÉðñçgW�ïUü
éú�&¯�Z�]!»Ø´éâ�w�C��ÈÅ��×ý$¾nµ(æ¦� DXÈ½�Ë�¿�VÊF5�eCtc�Ç.�·Zv�òí¸ÁM:�]t�@��Ú4 �÷Îh0�ñ'
@�,L�v¤aF)èªæ�¶ö3��gã4© ^¯§��Î��É1«��á¸^õO©/¥��äZZÿ��~;³�¾¨øÞºjã�~Ñ�ÁÆÀ�]#F�1óß�¥«Fîì¹Däox%¶�ÈóùlìØ��%�7¯k&9Ö¡³_Ý�«Ì'T?�q§T¹§0ÎB�W�ó|�+%�Ø°×<� y{Ö3aåáà��Ø�mäzd»·ÁõêÜ§�ç�¯ê ��g$ Ôd©�Xe«�n¥V"7�þùäÈ�üZþlNo

thanks Emanuele

base64 seems to not work even when enabled

I'm running version 0.1.5 of this filter. Here's my filter spec:

cipher {
  algorithm => "aes-256-cbc"
  # base64 seems to not actually work
  base64 => true
  mode => "encrypt"
  key => "redacted"
  iv => "redacted"
  source => "some_secret_plaintext"
  target => "some_secret_ciphertext"
  remove_field => ["message", "some_secret_plaintext"]
}

But the "some_secret_ciphertext" comes out in straight binary:

"some_secret_ciphertext":"PUÌþö#·\u000Fø%¹\f¸\u001Dt "

Consistently reproducible for me. I looked briefly at the code and at a glance it seemed like it should work. Not sure what's going wrong.

Plugin does not seem to support multiple sources

I'm not sure , but it looks like the plugin allows only to encrypt 1 source at a time since that source field is a string.

Cipher filter does not decrypt all messages

Hi,

I have problem with decrypting all sent messages via logstash. I've already worked out different algorithms. Errors that occurred:

[2019-05-02T13:57:55,623][WARN ][logstash.filters.cipher ] Exception catch on cipher filter {:event=>#LogStash::Event:0x52e0f039, :error=>#<OpenSSL::Cipher::CipherError: No message available>}
[2019-05-02T13:57:55,623][WARN ][logstash.filters.cipher ] Exception catch on cipher filter {:event=>#LogStash::Event:0x793feb6d, :error=>#<OpenSSL::Cipher::CipherError: last block incomplete in decryption>}
[2019-05-02T13:57:55,623][WARN ][logstash.filters.cipher ] Exception catch on cipher filter {:event=>#LogStash::Event:0x2d396dcb, :error=>#<OpenSSL::Cipher::CipherError: pad block corrupted>}
[2019-05-02T13:57:55,623][WARN ][logstash.filters.cipher ] Exception catch on cipher filter {:event=>#LogStash::Event:0x481459f2, :error=>#<OpenSSL::Cipher::CipherError: last block incomplete in decryption>}
[2019-05-02T13:57:55,676][WARN ][logstash.filters.cipher ] Exception catch on cipher filter {:event=>#LogStash::Event:0x2e2cb2bc, :error=>#<OpenSSL::Cipher::CipherError: pad block corrupted>}
[2019-05-02T13:57:55,676][WARN ][logstash.filters.cipher ] Exception catch on cipher filter {:event=>#LogStash::Event:0x66e491, :error=>#<OpenSSL::Cipher::CipherError: Cipher not initialized>}
The effect is that not all messages are correctly decoded.
You have an idea where these errors come from?

Note about v2.0.3 and prior PR/version compatibility

Want to make a note to anyone using a mixed footprint of cipher filter versions. Particularly patched v2.0.2 and older cipher filter encryptors with v2.0.3+ decryptors (and vice-versa)

Such a setup would be a unique edge case where you happen to be encrypting with logstash agents using the patched code from this original PR #3, rather than the official 2.0.2 (and older) plugin release (which is unlikely as it did not work properly anyway)

This particular change (i.e. moving from non-strict b64 en/decoding to strict) may cause the following error on either side (likely the decoding/decrypting side) in the logstash log, something to the effect of: ArgumentError: invalid base64.

The solution is to bring both sides of your encryption/decryption inline to the same cipher filter version. i.e. Both sides running v2.0.3+ or both sides running the patched code from PR #3

This really doesn't affect the official v2.0.2 and lesser released versions due to other bugs where it didn't work properly anyways.

Please see here, for line notes: 01311b3

Lastly, we could add an option to v2.0.3+ such as base64_strict: true|false to permit folks to toggle the use of strict vs non-strict base64 to permit such a mixed mode as described above, however I think this option would be pretty short-lived.

keys not tagged as "password", may leak in log messages

Keys are validated as "string", which does not protect them from being output in log messages:

https://github.com/logstash-plugins/logstash-filter-cipher/blob/master/lib/logstash/filters/cipher.rb#L41

Plugin does not seem to support Logstash 5.0

This is the output that I am receiving when trying to install in a RHEL7 running Logstash 5.0:

Installing logstash-filter-cipher
Plugin version conflict, aborting
ERROR: Installation Aborted, message: Bundler could not find compatible versions for gem "logstash-core-plugin-api":
  In snapshot (Gemfile.lock):
    logstash-core-plugin-api (= 2.1.16)

  In Gemfile:
    logstash-devutils (~> 1.1) java depends on
      logstash-core-plugin-api (~> 2.0) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-output-file (>= 0) java depends on
      logstash-core-plugin-api (< 2.99, >= 2.0.0) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-output-stdout (>= 0) java depends on
      logstash-core-plugin-api (< 2.99, >= 1.60.1) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-input-s3 (>= 0) java depends on
      logstash-mixin-aws (>= 0) java depends on
        logstash-core-plugin-api (<= 2.99, >= 1.60) java

    logstash-filter-cipher (>= 0) java depends on
      logstash-core-plugin-api (~> 1.0) java

    logstash-core-plugin-api (>= 0) java

Running `bundle update` will rebuild your snapshot from scratch, using only
the gems in your Gemfile, which may resolve the conflict.
Bundler could not find compatible versions for gem "logstash-core":
  In snapshot (Gemfile.lock):
    logstash-core (= 5.0.2)

  In Gemfile:
    logstash-core-plugin-api (>= 0) java depends on
      logstash-core (= 5.0.2) java

    logstash-filter-cipher (>= 0) java depends on
      logstash-core (< 2.0.0, >= 1.4.0) java

    logstash-core (>= 0) java

Running `bundle update` will rebuild your snapshot from scratch, using only
the gems in your Gemfile, which may resolve the conflict.
Bundler could not find compatible versions for gem "logstash":
  In Gemfile:
    logstash-filter-cipher (>= 0) java depends on
      logstash (< 2.0.0, >= 1.4.0) java
Could not find gem 'logstash (< 2.0.0, >= 1.4.0) java', which is required by gem 'logstash-filter-cipher (>= 0) java', in any of the sources.

Will this filter be compatible with Logstash 5?

Thanks,

thread not safe error: OpenSSL::Cipher::CipherError: Cipher not initialized

Products/Versions (include any installed plugins):
Logstash 6.7 (also reproduce on Logstash 6.5.1)
logstash-filter-prune 3.0.3
logstash-filter-mutate 3.5.0
logstash-filter-cipher 4.0.0

Operating system:
Linux Ubuntu

Description:
The following use of filters is triggering errors on the cipher plugin:

                 prune {
			blacklist_values => [ "message", "-"]
		}
		mutate {
			lowercase => ["message"]
		}
		cipher {
		    algorithm => "aes-256-cbc"
			cipher_padding => 1
			iv_random_length => 16
			key => "12345678901234567890123456789012"
			key_size => 32
			mode => "encrypt"
			source => "message"
			target => "encrypted_message"
			base64 => true
			max_cipher_reuse => 1
		}

A number of errror are seen such as:

[2019-06-19T14:48:52,234][WARN ][logstash.filters.cipher  ] Exception catch on cipher filter {:event=>#<LogStash::Event:0x57f70d58>, :error=>#<OpenSSL::Cipher::CipherError: Cipher not initialized>}
[2019-06-19T14:48:52,234][WARN ][logstash.filters.cipher  ] Exception catch on cipher filter {:event=>#<LogStash::Event:0x452f4cbd>, :error=>#<OpenSSL::Cipher::CipherError: No message available>}
[2019-06-19T14:48:52,284][WARN ][logstash.filters.cipher  ] Exception catch on cipher filter {:event=>#<LogStash::Event:0x7c5477d9>, :error=>#<NoMethodError: undefined method `update' for nil:NilClass>}

Important note: it is noticed the error is not seen is the number of worker is set to 1.
The test was done by setting the following value in logstash.yml file:

pipeline.workers:1

Hence the issue appears to be related to the fact we are not calling OpenSSL method in a thread safe way like the github issue linked above.

Steps to reproduce:
Setup a pipeline to parse logs as per tutorial below:
https://www.elastic.co/guide/en/logstash/current/advanced-pipeline.html
Then edit the conf file to use filters described above.
Run the pipeline.

Plugin does not seem to support Logstash 5.3

Hello,

do you have any plan to update this plugin to work with 5.3?
Regards
Alessandro

Add a key_file parameter to allow storing the key in a separate file

The company I work for would like to use the cipher plugin, but cannot do so for a couple reasons. The first is referenced by issue #1 (static IV), and the second is that we cannot store the cipher key in the Logstash configuration file.

Can you please allow a key_file parameter which would allow us to store the key in a separate file and out of version control? You can follow the lead of logstash-output-s3 and have a YAML file like this:

:key: "AAAAAAAAAAAAAAA"

An unexpected error occurred

An unexpected error occurred! {:error=>#<OpenSSL::Cipher::CipherError: Unsupported parameter: javax.crypto.spec.IvParameterSpec@74994263>, :backtrace=>["org/jruby/ext/openssl/Cipher.java:933:in reset'", "/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-filter-cipher-3.0.1/lib/logstash/filters/cipher.rb:190:in init_cipher'"

Any plan to merge outstanding existing pull request before starting yet another project?

Here is one I submitted last summer which has still to be evaluated/merged.

elastic/logstash-contrib#79