ConsumerSamplePrint#printExceptionCaught difftime -> diffTime
BrokerSamplePrint#printExceptionCaught difftime-> diffTime
DiskSamplePrint#printExceptionCaught difftime-> diffTime
BdbStoreSamplePrint#printExceptionCaught difftime-> diffTime

tube-server/test/ 

in .gitignore is not correct, It shold be

 tubemq-server/src/test/

Before processing the client request message, we need to check the current running status of the Broker to prevent the Broker from accepting new requests during the stop phase

The github button on homepage links to project of incubator-tubemq-website , I think it should link to TubeMQ

I upload three articles regarding to the TubeMQ Lib Interface Usage, the TubeMQ console operation guideand and the TubeMQ VS Kafka Performance Comparison Test Summary.

Further suggestions for translating is very much welcomed.

When the message is decoded, the number of ByteBuffer lists (listSize) and the length (len) value of each ByteBuffer may be negative values in the received RPC package. Negative value judgment is required, like: 

When Broker shuts down and goes offline, there is a certain chance that the offset of the consumer group is set to 0

in ProducerManager#push
wast time -> waste time

JVMClusterUtil#startup : regionservers -> regionServers
RowLock#getLock : lockid -> lockId

JIRA link - [INLONG-22] created by technoboy

I found some typoes about word available which show as avilable

JIRA link - [INLONG-41] created by tensorchen

JIRA link - [INLONG-23] created by gosonzhang

replace StringBuffer with StringBuilder

Same as TUBEMQ-7, but found in another place.

As for SDK, at present, the language are only limited to Java and C++. In the short and medium term，our efforts will be devoted to the core processing flow improvement, so it's very hard to rely on the existing effort to support the use of other languages for now.
However, TubeMQ project need more SDKs of corresponding language, such as Go, Python, PHP, C# language, for getting adopted by widely range of scenarios.

As for using tools, our environment is a DO separation mode. We may not have a good understanding of the tools you use everyday. Would you like to provide some corresponding ideas, or actually provide some tools to enrich TubeMQ's operation and maintenance in this area?

Go SDK see https://issues.apache.org/jira/browse/TUBEMQ-25

Recently, in actual testing and troubleshooting, we analyzed and organized the logic of file storage (org.apache.tubemq.server.broker.msgstore.disk), and found some problems that need to be optimized:

1. Because the data uses an out-of-date aging mechanism, the FileSegment file indexing method is not necessary，and this logic interferes will with the use of the system;

2. The FileReadView class is intended to construct a separate view of the read data processing, and at the same time control the read operation in the view. From the actual troubleshooting, this logic is not necessary, and no requests will read expired files, the class for read encapsulation is not needed;

3. The FileSegmentList class, from the test point of view, the use of List is not as good as the array method, and not faster, at the same time, this class is best to form a container without too much other business logic, so that the abstraction will be more suitable for its positioning.

I'll append my changes to improve them.

There are multiple 

System.out.Println

 codes in the project.

 This is not a good practice

I've noticed that the License in code files need modify, for example :

/*

• Tencent is pleased to support the open source community by making TubeMQ available.
  *
• Copyright (C) 2012-2019 Tencent. All Rights Reserved.
  *
• Licensed under the Apache License, Version 2.0 (the "License"); you may not use
• this file except in compliance with the License. You may obtain a copy of the
• License at
  *

I'll modify the files to use the standard Apache License declaration.

Javadoc is not right in class com.tencent.tubemq.corerpc.client.Callback

Current, the package name is "com.tencent.tubemq.", need change it to "org.apache.tubemq."

1. Adjust version format
2. Add mailist,scm,license,issueManagement

JIRA link - [INLONG-21] created by gosonzhang

JIRA link - [INLONG-18] created by technoboy

When the consumer obtains the messages, they does not carry the message source information, such as the message obtained from which broker and which partition. It is difficult to process the message statistics, so it is necessary to add the return of this information in the program implementation

We need to provide the project's communication methods in a centralized manner, including mailing lists, issue management entrances, etc., so that interested parties can join the project easily and quickly.

I will contribute the SDK of C + +, which is currently being sorted out. Without changing the server-side interaction protocol, the semantics will be the same as the Java implementation.

Task
1.io thread management
2. Connection interface
3. Connection pool management
4.IO Buffer
5.Future/Promise support
6.Codec interface and TubeMQ interactive encoding
7. Functionalization of service interface parameter settings
8.Future service interface
9.Master implementation, metadata acquisition and reading interface
10. Consumer realization
11. Producer Realization
12.Client configuration
13. Producer API package
14. Consumer API package

Plan
Phase 1 (Available)-Implement basic functions and ensure unit test coverage
RPC support
Message consumption (pull mode)
Write message (synchronous)
Connection pool implementation-automatic reconnection of broken links, automatic recovery of idle connections, shared according to sessionFactory
Test case and specification construction-

Phase 2 (Practical)-Realize the features of each function and be able to adapt to a certain level of pressure measurement
Asynchronous production 
Authentication and authentication && Prevent production and consumption around the master 
Accurately specify offset partition consumption 
Frequency control
Consumption of multiple topics in a single group 
Server filter consumption 

Phase 3 (Easy to use)-Continuous iteration, full-featured implementation, realizing the latest function points according to priority
Inactive for more than specified minutes, mainly at the production end, such as 3 minutes 
Automatically shield the dead pixels, detect the dead pixels through the algorithm, and automatically shield the fault Broker's data transmission 
TLS 
push consumption

Featrue
1.rpc development
Pre-research--asio development
Connection interface
Connection pool management-connection reuse, automatic reconnection of broken links, automatic recovery of idle connections, sharing according to sessionFactory
Streaming request support-based on serialNo
Future/Promise interface
Codec interface
Log
Config
Buffer

2. Service interface development
API service interface, such as heartbeat package variable function
Request Future/Promise function encapsulation
Config: client, consumer, producer, Master, rpc
Message id

3.Client
turn stop
Configuration input
Consumer API
Producer API

4.Master
Authentication
Prevent the production and consumption around the Master: get the token from the Master and update it to the local metadata
Metadata management: Topic, Group, Master
Master Heartbeat

5. Consumer
User interface
Heartbeat support
pull consumption
Consumption of multiple topics in a single group
Authentication
Precisely specify offset partition consumption

6. Producer
User interface
Simultaneous production
Asynchronous production
Load balancing algorithm: polling
Load balancing algorithm: hash

7. Advanced features
Master disconnects and reconnects, traversing dns to obtain Master host IP
Limiting
Effectively-Once
tls:asio+openssl
Filter consumption
Inactive for more than the specified minutes, mainly at the production end, for example 3 minutes
Fuse: Automatically shield the dead spots, detect the dead spots through the algorithm, automatically shield the fault Broker's data transmission
Reporting time-consuming, failure; static weight (number of partitions), dynamic weight
Connection pool management-automatic recovery of idle connections, shared according to sessionFactory
Push consumptio

When I compile the project and report thes errors:

After the log file is successfully initialized, the instance should be written to the dataStores directly in the task, and there is no need to write the result by returning in loadStoresInParallel(), in order to improve the overall task and release. 

How the user can join the community, any mail list , QQ Group, Slack or what every .

How developers can join and see what can help to contribute.

Need a document guide them.

We use JDK 1.7 for compilation, but iterator method in MapBackedSet has compile error due to using 1.8 implementation.

1. 'String.valueOf()'

String strCallbackFun = req.getParameter("callback");
if ((TStringUtils.isNotEmpty(strCallbackFun))
&& (strCallbackFun.length() <= TBaseConstants.META_MAX_CALLBACK_STRING_LENGTH)
&& (strCallbackFun.matches(TBaseConstants.META_TMP_CALLBACK_STRING_VALUE))) {
    strCallbackFun = String.valueOf(strCallbackFun).trim();
}

com.tencent.tubemq.server.master.web.action.screen.Webapi#execute

2. {String}.substring()

if (path.startsWith("/")) {
    path = path.substring(1, path.length());
}

com.tencent.tubemq.server.master.web.simplemvc.RequestContext#normalizePath

3.unuse init boolean

private boolean isOverTLS = false;

com.tencent.tubemq.corerpc.netty.NettyRpcServer  isOverTLS 

PR #3

Develop a standalone mode to start tubemq and debug tubemq code locally

ref https://issues.apache.org/jira/browse/INFRA-19681

method jsEncode and jsDecode in BaseResult is using StringBuffer.

A bug for groupName check!

Some letters are wrong in http_access_API_definition.md

In the shutdown () of TubeSingleSessionFactory and SimpleMessageProducer, the assignment has been compared through the AtomicBoolean variable to avoid repeated releases. This definition is redundant.

I noticed some screenshots in https://tubemq.apache.org/en-us/docs/tubemq_user_guide.html contain Chinese characters.

Better to change them, which would be more friendly to users/developers whose mother language is not Chinese.

The project's official website is still not available: http://tubemq.incubator.apache.org, need to get done ASAP.

In tubemq-cilent module, file path org/tencent/apache/client is not match with package name org.apache.tubemq.client .

Some enum classes are implement by annoymouse inner class, like EventStatus .
I think use a constructor with necessary parameters is a better and more graceful way.

The event item is printed to show the removed events, if there is no such consumer event at the time of searching, it should not to print null log

In BrokerConfManage :
method isPrimaryNodeActived should be named isPrimaryNodeActive

As shown below: The setTimeoutTime function is to update the timeout period periodically. It is named updTimeoutTime, and it is more appropriate to put the time inside the function.

1. add NOTICE file
2. adjust LICENSE file content
3. adjust pom.xml, not check the NOTICE,LICENSE's RAT

JIRA link - [INLONG-24] created by gosonzhang

There are already several existing MQs and some ecosystem are built around them. TubeMQ should work with existing tools and ecosystem which will enhance the early adoption of TubeMQ.

correct DefaultLoadBalancer#balance rejGroupClientINfoMap -> rejGroupClientInfoMap

There are many users who use the C/C++ language, but it is worth noting that more and more businesses use the Go language, and it is not a long-term way only having C/C++ SDK, many users like the pure Go SDK.

 I will contribute the SDK of Go. Without changing the server-side interaction protocol, the semantics will be the same as the Java and C++ implementation.

1. Multiplexed connection pool
2. Connection interface
3. Codec interface and TubeMQ interactive encoding
4. Selector
5. Master implementation, metadata acquisition and reading interface
6. Consumer realization
7. Client configuration
8. Consumer API

As shown below: If the conumser is registered with the broker and exits abnormally, the timer will not expire.

Recently, the fastjson security vulnerability problem has again emerged. This component is also referenced in the TubeMQ code. In the near future, it is planned to replace fastjson with gson.

The value of DeletePolicy is a structured data content. When inputting this data, content check should be added to avoid inconsistency between the set value and the actual effective situation