Comments (6)
@caohuigh 我这边试了下,好像没遇到你说的报错,这个配置你试下?
indexer-kafka-test.yml
inputs:
- Stdin:
codec: json
filters:
- Convert:
fields:
server:
setto_if_fail: 0
to: int
id:
setto_if_fail: 0
to: int
outputs:
- Stdout: {}
gohangout --config indexer-kafka-test.yml
{"server":666,"ip":"1.2.3.4","id":100}
{"server":666,"ip":"1.2.3.4","id":100,"@timestamp":"2019-05-09T18:21:38.904303797+08:00"}
另外,panic: interface conversion: interface {} is string, not map[interface {}]interface {}
这个错误更像是yaml格式(类似前排空格缩进)问题,可以查下你的yaml文件格式是不是有问题。
from gohangout.
@caohuigh 使用Convert转成int类型. https://github.com/childe/gohangout/#convert
from gohangout.
@childe 谢谢回复,您指的是使用FILTER里面的convert?我主要是想把string类型转成int64,格式是怎样的,可以给个例子吗?
from gohangout.
Convert:
fields:
server:
setto_if_fail: 0
to: int
ip:
to: int
setto_if_fail: 0
from gohangout.
不好意思,是我没说清楚引起误解了,我想请教的是Convert应该放在什么位置?是放在filters下面吗?
比如这样:
inputs:
- Kafka:
topic:
... ...
filters:
- Convert:
fields:
server:
to: int
remove_if_fail: false
id:
to: int
remove_if_fail: false
outputs:
- Clickhouse:
... ...
但是我这样写运行的时候却报错了:
I0509 15:52:21.857921 187339 filter.go:72] filter type: Convert
panic: interface conversion: interface {} is nil, not map[interface {}]interface {}
goroutine 27 [running]:
github.com/childe/gohangout/filter.BuildFilterBoxes(0xc0001b5a40, 0x9f6740, 0xc0001e9630, 0x1, 0x0, 0x0)
我在output中的fields中是这样写的:
fields: ['server', 'ip', 'id']
我也试过:
fields: [server, 'ip', id]
但无论使用fields的哪种写法,结合上面的Convert时都会报错,是我哪里格式写错了吗?
from gohangout.
@zhoufwind
非常感谢您的建议,我检查了一下,确实在filters字段中整体少了一个空格,导致一直看不出来,已经测试入库成功了,谢谢!
from gohangout.
Related Issues (20)
- Improvement: build multi-arch official image
- Skip MATERIALIZED columns when writing ClickHouse table. HOT 1
- Logging without becoming coupled to a particular logging implementation HOT 3
- 能否实现kafka数据的可靠消费(at least once)? HOT 7
- es8 下没有对 es 返回做429 状态码做正确处理 HOT 2
- 编写插件加载失败 ould not open /zcola/pulsar_input.so: plugin: not implemented HOT 21
- 读取pattern-paths报错 build grok filter error HOT 4
- 没有正常解析数据 HOT 1
- 1.10 版本消费kafka 一段时间后不继续消费报错 HOT 21
- 关于字段过长需要截断 HOT 2
- unknownType main.Coordinates 写了一个插件解析geoip2 mmdb数据库,stdout 输出正常,es 确写入报错 HOT 2
- 支持将Json Array 转换成多行到CK吗? HOT 1
- JSON解析能否跳过不规则字符? HOT 2
- 多个Filters不生效 HOT 2
- kafka多个主题,插入多个clickhouse表 HOT 1
- 如何从kafka读取,输出到starrocks中 HOT 1
- hangout 不定时crash HOT 2
- make Error HOT 2
- 想要将syslog中某些可能为空的字段设置一个默认值,但一直不成功 HOT 1
- kafka 配置 ipv6 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gohangout.