Comments (11)
When broker gets a message published by client, it stores the message in a commitlog, which i believe, is the reason you are seeing the increase in memory usage as the number message published increase!
Growth in memory usage as the number of messages increase is expected till a certain limit. You can control the maximum size of the segment in commitlog by max_segment_size
& max_segment_count
in rumqttd.toml
. ( commitlog is basically collection of those segments )
from rumqtt.
I look at these parameters, set the maximum number, the new package will not be accepted, is rejected; So how do you clear the old data to make room for the new data
from rumqtt.
if there is no space, older segments are dropped automatically by the broker to make space for new data! can you provide rumqttd config and the client code you are using to we can debug this further?
from rumqtt.
BTW, it would be also helpful if you paste the failure logs, like wdym by package being rejected, which packet it was and what did rumqttd log the reason of rejecting the packet.
from rumqtt.
This is my toml file
`[router]
id = 0
max_connections = 10010
max_outgoing_packet_count = 200
max_segment_size = 104857600
max_segment_count = 10
[v5.1]
name = "v5-1"
listen = "0.0.0.0:18804"
next_connection_delay_ms = 1
[v5.1.connections]
connection_timeout_ms = 60000
max_payload_size = 20480
max_inflight_count = 100
Here is my broker and client codeοΌ
thread::spawn(||{
let mut broker = Broker::new(configs);
broker.start().unwrap();
println!("start");
});
thread::sleep(Duration::from_secs(1));
let mut mqttoptions = MqttOptions::new("test", "127.0.0.1", 18804);
mqttoptions
.set_keep_alive(Duration::from_secs(5));
let (client, mut eventloop) = AsyncClient::new(mqttoptions,5 );
task::block_on(client.subscribe("/upload/data", QoS::AtLeastOnce)).unwrap();
// thread::spawn(move||
loop{
let event_result = task::block_on(eventloop.poll());
match event_result {
Ok(event)=>{
match event{
Event::Incoming(incoming)=> {
if let Incoming::Publish(publish) = incoming.clone() {
println!("message {:?}", publish.topic);
let res=task::block_on(handel_topic(publish, "test1".to_string(), client.clone()));
}
}
Event::Outgoing(outgoing)=>{
println!("outgoing {:?}",outgoing)
}
}
}
Err(e)=>{
println!("err {:?}",e);
}
}
}`
from rumqtt.
When the program reaches this step, "let event_result = task::block_on(eventloop.poll()); If the client continues to send messages, the memory will continue to expand
from rumqtt.
It seems to have been successfully solved, the method is that I set the toml file parameter max_segment_size = 10000,
and then the program memory will not expand, I go to apply this configuration to my program. I'll see if it affects the content and quality of the message
from rumqtt.
That's good to know! Closing this as it isn't an bug / issue ( as described in the comments ). Feel free to comment if you wish!
Btw, just curious why you are using Async client instead of sync Client if you want to block on everything instead of calling .await
?
Thanks for reporting π
from rumqtt.
Thank you for your reminder to modify toml!
Asynchronous is chosen to avoid unnecessary waiting and blocking on synchronous clients, aren't asynchronous clients more friendly
from rumqtt.
Asynchronous is chosen to avoid unnecessary waiting and blocking on synchronous client
Yup! But in your code, you are using task::block_on
which blocks right? Async will be beneficial if u use .await
instead of block_on right?
e.g. client.subscribe("topic").await?
note: that ? can be replaced with .unwrap()
as per ur provided example
from rumqtt.
Yes, we do! Thank u. Those are two things I can improve on
from rumqtt.
Related Issues (20)
- rumqttc: getting incorrect unsolicited acks error logs
- macOS ci is broken HOT 2
- rumqttc - AUTH packet support HOT 13
- rumqttc - ack with reason code HOT 3
- rumqttc: Could publish/subscribe methods directly return the msg id? HOT 2
- Does `MqttOptions` `request_channel_capacity` field do anything? HOT 3
- Keep the ACK ordering when using manual ACK HOT 2
- Unexpected PUBACK Packet Sent to Reconnected Broker HOT 1
- rumqttc - How to connect to Mosquitto test broker with TLS HOT 2
- Connect with session_expiry_interval for MQTT v5 HOT 4
- client: how to reconnect with subscriptions? HOT 1
- quic protocol HOT 3
- rumqttc - Event ordering of puback is incorrect
- In mqttc v5, restore session only if session is resumed HOT 1
- InvalidCertificate(BadSignature) when using TLS with custom root certificate HOT 1
- Add customization options when using native-tls
- rumqttc: Reduce memory usage of MqttState
- Option to set the `tcp-nodelay` flag used for the connection from the MQTT client to the broker
- rumqttc: log `Outgoing::Connect` HOT 1
- Unexpected Reconnect on MqttState errors HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. πππ
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google β€οΈ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rumqtt.