Giter VIP home page Giter VIP logo

quickfix-j / quickfixj Goto Github PK

View Code? Open in Web Editor NEW
943.0 93.0 609.0 42.73 MB

QuickFIX/J is a full featured messaging engine for the FIX protocol. - This is the official project repository.

Home Page: http://www.quickfixj.org

License: Other

Java 91.84% XSLT 2.93% HTML 5.16% CSS 0.05% Shell 0.01% PLpgSQL 0.01% Batchfile 0.01%
quickfix quickfixj fixprotocol java protocol cryptocurrency finance financial-information-exchange financial-markets fintech

quickfixj's Introduction

QuickFIX/J

Java CI CodeQL Maven Central

This is the official QuickFIX/J project repository.

intro

QuickFIX/J is a full featured messaging engine for the FIX protocol (FIX versions 4.0 - 5.0SP2/FIXT1.1 and FIXLatest). It is a 100% Java open source implementation of the popular C++ QuickFIX engine.

The Financial Information eXchange (FIX) protocol is a messaging standard developed specifically for the real-time electronic exchange of securities transactions. FIX is a public-domain specification owned and maintained by FIX Protocol, Ltd (FPL).

For more information see the project website at http://www.quickfixj.org.

release notes

Check out the wiki: https://github.com/quickfix-j/quickfixj/wiki

questions

For asking questions please either use the mailing list https://lists.sourceforge.net/lists/listinfo/quickfixj-users or ask on Stack Overflow https://stackoverflow.com/questions/ask?tags=quickfixj .

issues

Please report issues here: https://github.com/quickfix-j/quickfixj/issues

security

QuickFIX/J welcomes and appreciates responsible disclosure. Contributors are given appropriate credit in release notes and Git logs.

For security issues in QuickFIX/J itself contact the project maintainer: christoph.john-at-macd.com

For security issues in libraries used by QuickFIX/J contact the relevant project team (e.g. for Apache MINA: https://www.apache.org/security/ ). If you feel they are particularly exploitable via QuickFIX/J also feel free to follow up with the project maintainer as above so that we upgrade to the new version in a timely fashion.

Once a security issue is fixed in QuickFIX/J it will be communicated via the user mailing list and other appropriate channels.

contributions

Pull requests are always welcome! Best is if you added a unit test to show that a certain bug has been fixed or a new feature works as expected.

build instructions

Fastest: clone the repo and issue the following command.

$ mvnw clean package -Dmaven.javadoc.skip=true -DskipTests -PskipBundlePlugin,minimal-fix-latest

Slower: if you only want to skip the acceptance test suite:

$ mvnw clean package -Dmaven.javadoc.skip=true -DskipAT=true -PskipBundlePlugin,minimal-fix-latest

Slow: if you want to run all tests:

$ mvnw clean package -Dmaven.javadoc.skip=true -PskipBundlePlugin,minimal-fix-latest

NB: If you want to use the resulting JARs in an OSGi environment you'll have to omit the -PskipBundlePlugin option.

importing the project into the IDE

When the project is first created, it will not have the generated message classes and compile errors will occur! Best is to compile once on the command line before importing the project into the IDE.

If the IDE reports some errors after the compilation with mvnw clean package, try to use mvnw clean install, like:

$ mvnw clean install -Dmaven.javadoc.skip=true -DskipTests -PskipBundlePlugin,minimal-fix-latest

configuration options

https://rawgit.com/quickfix-j/quickfixj/master/quickfixj-core/src/main/doc/usermanual/usage/configuration.html

basics

example applications

QuickFIX/J includes some example applications in the quickfixj-examples module. Moreover, here are some links to example applications:

Examples by Geoffrey Gershaw: https://github.com/ggershaw/Examples

Examples from QuickFIX/J Spring Boot Starter: https://github.com/esanchezros/quickfixj-spring-boot-starter-examples

If you would like to be added to this list, please open a PR with the changes.

Creating a QuickFIX/J application

Implement the quickfix.Application interface.

By implementing these interface methods in your derived class, you are requesting to be notified of events that occur on the FIX engine. The function that you should be most aware of is fromApp.

Here are explanations of what these functions provide for you.

onCreate is called when QFJ creates a new session. A session comes into and remains in existence for the life of the application. Sessions exist whether or not a counter party is connected to it. As soon as a session is created, you can begin sending messages to it. If no one is logged on, the messages will be sent at the time a connection is established with the counterparty.

onLogon notifies you when a valid logon has been established with a counter party. This is called when a connection has been established and the FIX logon process has completed with both parties exchanging valid logon messages.

onLogout notifies you when an FIX session is no longer online. This could happen during a normal logout exchange or because of a forced termination or a loss of network connection.

toAdmin provides you with a peek at the administrative messages that are being sent from your FIX engine to the counter party. This is normally not useful for an application however it is provided for any logging you may wish to do. Notice that the quickfix.Message is mutable. This allows you to add fields to an administrative message before it is sent out.

toApp is a callback for application messages that are being sent to a counterparty. If you throw a DoNotSend exception in this method, the application will not send the message. This is mostly useful if the application has been asked to resend a message such as an order that is no longer relevant for the current market. Messages that are being resent are marked with the PossDupFlag in the header set to true; If a DoNotSend exception is thrown and the flag is set to true, a sequence reset will be sent in place of the message. If it is set to false, the message will simply not be sent. Notice that the quickfix.Message is mutable. This allows you to add fields to an application message before it is sent out.

fromAdmin notifies you when an administrative message is sent from a counterparty to your FIX engine. This can be useful for doing extra validation on Logon messages such as for checking passwords. Throwing a RejectLogon exception will disconnect the counterparty.

fromApp is one of the core entry points for your FIX application. Every application level request will come through here. If, for example, your application is a sell-side OMS, this is where you will get your new order requests. If you were a buy side, you would get your execution reports here. If a FieldNotFound exception is thrown, the counterparty will receive a reject indicating a conditionally required field is missing. The Message class will throw this exception when trying to retrieve a missing field, so you will rarely need the throw this explicitly. You can also throw an UnsupportedMessageType exception. This will result in the counterparty getting a reject informing them your application cannot process those types of messages. An IncorrectTagValue can also be thrown if a field contains a value that is out of range or you do not support.

The sample code below shows how you might start up a FIX acceptor which listens on a socket. If you wanted an initiator, you would simply replace the acceptor in this code fragment with a SocketInitiator. ThreadedSocketInitiator and ThreadedSocketAcceptor classes are also available. These will supply a thread to each session that is created. If you use these you must make sure your application is thread safe.

import quickfix.*;
import java.io.FileInputStream;

public class MyClass {

  public static void main(String args[]) throws Exception {
    if (args.length != 1) return;
    String fileName = args[0];

    // FooApplication is your class that implements the Application interface
    Application application = new FooApplication();

    SessionSettings settings = new SessionSettings(new FileInputStream(fileName));
    MessageStoreFactory storeFactory = new FileStoreFactory(settings);
    LogFactory logFactory = new FileLogFactory(settings);
    MessageFactory messageFactory = new DefaultMessageFactory();
    Acceptor acceptor = new SocketAcceptor
      (application, storeFactory, settings, logFactory, messageFactory);
    acceptor.start();
    // while(condition == true) { do something; }
    acceptor.stop();
  }
}

Receiving messages

Most of the messages you will be interested in looking at will be arriving in your overloaded fromApp method of your application. You can get fields out of messages with different degrees of type safety. The type in question here is the FIX message type.

When the application passes you a Message class, the Java type checker has no idea what specific FIX message it is, you must determine that dynamically. There is, however, a way we can make Java aware of this type information.

Keep in mind that all messages have a header and a trailer. If you want to see fields in them, you must first call getHeader() or getTrailer() to get access to them. Otherwise you access them just like in the message body.

QuickFIX/J has message classes that correlate to all the messages defined in the spec. They are, just like the field classes, generated directly off of the FIX specifications. To take advantage of this, you must break the messages out with the supplied MessageCracker.

import quickfix.*;
import quickfix.field.*;

public void fromApp(Message message, SessionID sessionID)
      throws FieldNotFound, UnsupportedMessageType, IncorrectTagValue {

  crack(message, sessionID);
}

public void onMessage(quickfix.fix42.NewOrderSingle message, SessionID sessionID)
      throws FieldNotFound, UnsupportedMessageType, IncorrectTagValue {

  ClOrdID clOrdID = new ClOrdID();
  message.get(clOrdID);

  ClearingAccount clearingAccount = new ClearingAccount();
  message.get(clearingAccount);
}

public void onMessage(quickfix.fix42.OrderCancelRequest message, SessionID sessionID)
      throws FieldNotFound, UnsupportedMessageType, IncorrectTagValue {

  ClOrdID clOrdID = new ClOrdID();
  message.get(clOrdID);

  // compile time error!! field not defined for OrderCancelRequest
  ClearingAccount clearingAccount = new ClearingAccount();
  message.get(clearingAccount);
}

In order to use this you must use the MessageCracker as a mixin to your application. This will provide you with the crack method and allow you to overload specific message functions.

Any function you do not overload will by default throw an UnsupportedMessageType exception

Define your application like this:

import quickfix.Application;
import quickfix.MessageCracker;

public class MyApplication extends MessageCracker implements quickfix.Application {
    public void fromApp(Message message, SessionID sessionID)
            throws FieldNotFound, UnsupportedMessageType, IncorrectTagValue {
        crack(message, sessionID);
    }

    // Using annotation
    @Handler
    public void myEmailHandler(quickfix.fix50.Email email, SessionID sessionID) {
        // handler implementation
    }

    // By convention (notice different version of FIX. It's an error to have two handlers for the same message)
    // Convention is "onMessage" method with message object as first argument and SessionID as second argument
    public void onMessage(quickfix.fix44.Email email, SessionID sessionID) {
        // handler implementation
    }
}

If you'd rather use composition rather than inheritance of the MessageCracker you can construct a message cracker with a delegate object. The delegate message handler methods will be automatically discovered.

Message crackers for each FIX version are still generated for backward compatibility but it's more efficient to define the specific handlers you need.

The generated classes define handlers for all messages defined by that version of FIX. This requires the JVM to load those classes when the cracker is loaded. Most applications only need to handle a small subset of the messages defined by a FIX version so loading all the messages classes is excessive overhead in those cases.

Functional interfaces for receiving messages

If you prefer using lambda expressions in handling received messages, then ApplicationFunctionalAdapter or ApplicationExtendedFunctionalAdapter can be used to register reactions to events the application is interested in.

They also allow registering the interests in a given message type in a type-safe manner.

import quickfix.ApplicationFunctionalAdapter;
import quickfix.SessionID;

public class EmailForwarder {
    public void init(ApplicationFunctionalAdapter adapter) {
        adapter.addOnLogonListener(this::captureUsername);
        adapter.addFromAppListener(quickfix.fix44.Email.class, (e , s) -> forward(e));
    }

    private void forward(quickfix.fix44.Email email) {
        // implementation
    }

    private void captureUsername(SessionID sessionID) {
        // implementation
    }
}

ApplicationFunctionalAdapter and ApplicationExtendedFunctionalAdapter support multiple registration to the same event, and the registered callbacks are invoked in the FIFO manner.

However FIFO cannot be guaranteed between registration with specific message type (e.g. quickfix.fix44.Email) and that without specific message type. For example, there is no invocation order guarantee between the following two callbacks:

    adapter.addFromAppListener((e , s) -> handleGeneral(e));

    adapter.addFromAppListener(quickfix.fix44.Email.class, (e , s) -> handleSpecific(e));

Sending messages

Messages can be sent to a counter party with one of the static Session.sendToTarget methods. This method has several signatures. They are:

package quickfix;

public static boolean sendToTarget(Message message)
  throws SessionNotFound

public static boolean sendToTarget(Message message, SessionID sessionID)
  throws SessionNotFound

public static boolean sendToTarget
  (Message message, String senderCompID, String targetCompID)
  throws SessionNotFound

The highly recommended method is to use the type safe message classes. This should typically be the only way you should ever have to create messages.

Here the constructor takes in all the required fields and adds the correct MsgType and BeginString for you. What's more, by using the set method instead of setField, the compiler will not let you add a field that is not a part of a OrderCancelRequest based on the FIX4.1 specs. Keep in mind that you can still use setField if you want to force any field you want into the message.

import quickfix.*;

void sendOrderCancelRequest() throws SessionNotFound {
  quickfix.fix41.OrderCancelRequest message = new quickfix.fix41.OrderCancelRequest(
	new OrigClOrdID("123"),
	new ClOrdID("321"),
	new Symbol("LNUX"),
	new Side(Side.BUY));

  message.set(new Text("Cancel My Order!"));

  Session.sendToTarget(message, "TW", "TARGET");
}

QuickFIX/J Runtime

This project builds artefacts for the standard published FIX specification versions from FIX 4.0 to FIX Latest.

  • quickfixj-messages-fix40
  • quickfixj-messages-fix41
  • quickfixj-messages-fix42
  • quickfixj-messages-fix43
  • quickfixj-messages-fix44
  • quickfixj-messages-fix50
  • quickfixj-messages-fix50sp1
  • quickfixj-messages-fix50sp2
  • quickfixj-messages-fixlatest
  • quickfixj-messages-fixt11
  • quickfixj-messages-all - includes all of the above

These artefacts are test dependencies of quickfixj-core. They are not specified as runtime dependencies, this makes it easier to customise QuickFIX/J deployments.

If you have no need to customise a FIX integration then you can use the org.quickfixj artefacts built by this project. Simply include them as dependencies of your application.

Artefacts for unused FIX specification versions can be omitted from your runtime. Many integrations will not require quickfixj-messages-all and need only depend on artefacts for a subset of the FIX standard versions. Please note that FIX Protocol versions 5.0 and later depend on quickfixj-messages-fixt11 which provides the implementation for the FIXT1.1 transport messages.

Many integrations require specialisation of the FIX Messages, Components and/or Fields. This is accomplished by building and using custom artefacts. Please see Customising QuickFIX/J for more detail.

Application Dependencies for QuickFIX/J Messages Build

image info

image info

Application Dependencies for Custom Messages Build

image info

image info

quickfixj's People

Contributors

alexwibowo avatar amichair avatar arthurm1 avatar busy-spin avatar charlesbr1 avatar chrjohn avatar david-gibbs-ig avatar dependabot[bot] avatar enigma13090 avatar felipewind avatar gbirchmeier avatar jklmrk avatar jonfreedman avatar knutae avatar lburgazzoli avatar manureno avatar melowe avatar mmitya avatar mrbald avatar philipwhiuk avatar raipc avatar rockdreamer avatar rockmanjoe64 avatar snyk-bot avatar soliad avatar sometest1234 avatar the-thing avatar troshanin avatar vinipx avatar wojciechzankowski avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

quickfixj's Issues

SessionStateListener not notified about initial connection failures

Describe the bug
If a Session wholly fails to initialise (never gets as far as a TCP connection) nothing is notified to its SessionStateListener

To Reproduce

  • Add a SessionStateListener to to a Session configured such that it will never connect
  • Observe the event log has something like 'java.net.ConnectException during connection to /localhost:80: java.net.ConnectException: Connection timed out (Next retry in 30000 milliseconds)'

Expected behavior
Something to be notified to your listener (onDisconnect ?)

system information:

  • QFJ 2.1.1

Additional context
The exception shown above is sent to the qfj Log framework then swallowed by quickfix.mina.initiator.IoSessionInitiator.ConnectTask#handleConnectException

Listener onDisconnect is the only viable existing method that would be suitable although I don't know if it's necessarily the right choice. That rather depends on how it used for other categories of first-time initialisation error (i.e. an SSL error or FW issue) that manifest after TCP is connected.

the ESTAB tcp connection remains even after closed initiator

Describe the bug
the acceptor written in java is running on redhat vm, and the initiator written in c# is running in local win10 pc;

after suceesfully exchange messages, I left the initiator there without any message exchange for about half an hour, then I closed the initiator, but the tcp connection remains ESTAB, as showing in below screenshot:
image

and when I try to run the initiator again, it just cannot logon, toadmin cannot get a response,
and the acceptor keep printing:
"2020-03-17 16:25:15.617 INFO 13057 --- [ NioProcessor-9] q.mina.acceptor.AcceptorIoHandler : MINA session created: local=/xxxxxxx:9870, class org.apache.mina.transport.socket.nio.NioSocketSession, remote=/xxxxxx:30986"
generated a lot TIME_WAIT tcp connections
image

it looks like something has blocked the new connection, obviously the ESTAB Tcp connection must have onhold some resources I guess

To Reproduce
as described above

Expected behavior
the tcp connection should disconnected after closing the initiator

system information:

  • OS: [e.g. Linux]: Red Hat Enterprise Linux Server 7.2 (Maipo)
  • Java version: openjdk version "1.8.0_232"
  • QFJ Version: quickfix-core 2.1.0 (quickfixj-spring-boot-starter 2.4.1)

The bug of generateSequenceResetIfNeeded in resendMessages method

image

”generateSequenceResetIfNeeded“method:What I understand is that when the retrieved messages to be processed are less than the endseqno, for example, messages from 2 to 10 are expected to be sent, but only 2 to 6 are retrieved in persistence, and the sequence number needs to be reset to 11;However, there is a bug in the current code logic. When message 5 in 2 to 6 is an app message and 6 is an admin message, a reset message with sequence number 2 will be sent. However, the seqno expected by the other party has been reset to 6 when sending the app message with sequence number 5, which will cause the sequence number to be too low. I think the newbegin variable should be the sequence number expected by the current party, so it should be changed to Newbegin is defined above the for loop and is assigned the current sent message sequence number + 1 at appmessage justsent = true

ResetOnLogon/ResetOnLogout/ResetOnDisconnect fails when PersistMessages is set to 'N'

Describe the bug
ResetOnLogon/ResetOnLogout/ResetOnDisconnect fails when PersistMessages is set to 'N'. It throws an IOException "Invalid object name messages"

at quickfix.JdbcStore.reset(JdbcStore.java:195)

To Reproduce
Set ResetOnLogon/ResetOnLogout/ResetOnDisconnect to Y and PersistMessages to N. When Logon/Logout/Disconnect event occurs, the session tries to reset state, which in turn tries to reset the message store. On JdbcStore, this results in trying to delete from messages table, before updating the sessions table. If PersistMessages is set to N, no messages table is created.

Expected behavior
If PersistMessages is set to N, no attempt should be made to delete from messages table.

system information:

  • OS: Linux
  • Java version JDK8
  • QFJ Version 2.2.0

Additional context
Either update https://www.quickfixj.org/usermanual/2.1.0/usage/configuration.html to clearly mention which settings can be used in conjunction with another. Or pull delete messages logic from lines 182-185 in JdbcStore.java (QFJ Version 2.2.0) into a separate method deleteMessages and call it from line 2661 of Session.java with a check if persistMessages is true around it.

When and How are sessions closed?

Hi chrjohn,

I'm a new person using quick fix from China, and I have a question about create and close session.
In createSessions method of AbstractSocketAcceptor.java, you create a session using Session Factory, I think it's good, the Session implements Closeable interface, however, you don't put 'sessionFactory.create(sessionID, settings)' into try source, so I want to know When and how are sessions closed.
My app prints 'Too many open files' exception every second in QFJ Timer thread, so I wonder if it's because session isn't closed correctly.

Or where is there opening file operation in 'QFJ Timer' thead? sorry, I search but no result.

Looking forward to your reply & thanks
Tim

ContinueInitializationOnError only works for Initiators

Describe the bug
We have a setting ContinueInitializationOnError which could be used when a single session should not prevent the whole connector (with multiple sessions) from starting. However, it is only implemented for Initiators.

private void createSessions() throws ConfigError, FieldConvertError {
final SessionSettings settings = getSettings();
boolean continueInitOnError = false;
if (settings.isSetting(SessionFactory.SETTING_CONTINUE_INIT_ON_ERROR)) {
continueInitOnError = settings.getBool(SessionFactory.SETTING_CONTINUE_INIT_ON_ERROR);
}

Expected behavior
This should also work for Acceptors.

Do not log cleartext password on logon failure

Describe the bug
Full FIX message with cleartext password is logged on logon failure.

Expected behavior
Either do not log the FIX message, or remove cleartext password from the message.

Expose sequence recovery data externally via the SessionStateListener interface

Today, there's no way of knowing what's the current sequence recovery state of the engine. We're are currently developing a client application that needs to react on ResendRequest sent to the target and SequenceReset messages (for hard resets) received from it.

So, my suggestion is to enhance the SessionStateListener with the following methods.

void onResendRequestSent(int beginSeqNo ,int endSeqNo, int currentEndSeqNo) 

This method will called when the send issues a ResendRequest to the target, populated by the values of the reset -equest range

void onSequenceResetReceived(int newSeqNo, boolean gapFillFlag)

This method will be called when we recieve a SequenceReset messages, mostly here in order to allow the sender side tobe made aware of hard-resets (when the gapFillFlag is false)

void onResendRequestSatisfied(int beginSeqNo ,int endSeqNo)

This method will be called when the quickfix engine assumes the gap has been satisfied (meaning, sequence recovery has completed)

I'd be willing to contribute a PR, should you approve this request.

Graceful disconnect functionality

There's no straightforward way to tell a session "Logout and then disconnect"

If you call 'disconnect()' we just disconnect - we don't try to send a logout let alone wait for a response - we take a reason but we don't send it to the client. This is seen in #244 and #101

Ideally there would be a method similar to:

Future<LogoutResponse> logoutAndDisconnect(int waitForResponseTimeout) {}

which:

  • sent a logout
  • optionally waited for a response (if timeout > 0)
  • disconnected

This would be a high level call which abstracted away all the interaction waiting for the logout response. The future would provide a way of monitoring it if desired.

Alternatives/work-arounds:

  • User implementation of behaviour in app-code
  • Just calling disconnect and hoping the other side behaves reasonably

`DataDictionary.getVersion()` ignores service pack

Describe the bug
dataDictionary.getVersion() ignores the XML's @servicepack attribute. A FIX.5.0.SP2 dictionary XML will return "FIX.5.0". This feels wrong ...

To Reproduce
This from your own quickfix.MessageTest: ...

    final DataDictionary dataDictionary = new DataDictionary("FIX44.xml");
    final Message message = new DefaultMessageFactory()
            .create(dataDictionary.getVersion(), "D");
    message.fromString(expectedMessageString, dataDictionary, false);

... is a useful shorthand (creating/using the correct MessageFactory requiring only a vendor-provided dictionary). It is likely to cause bugs if it was FIX.5.0SP2. If message is passed to downstream code, it is highly likely somewhere it will be tested/casted with

if (message instanceof quickfix.fix50sp2.ExecutionReport) ...

and fail since quickfix.fix50.ExecutionReport is binary incompatible.

Expected behavior
OPTION1 getVersion() to return "FIX.5.0SP2" where appropriate.
OPTION2 add String getFullVersion()
OPTION3 Explicitly expose int getServicePack()

system information:

  • QFJ Version 2.1.1

I'm happy to PR this little change if stakeholders approve a design option.

Add TLS SNI support

Is your feature request related to a problem? Please describe.
Quickfix/J now supports SSL, which is great, but to fully leverage this when running a mutli-client server in a managed (Kuberenetes/Traefik2) environment (and many others), one can't always allocate a new port for each new session, and instread 3rd+ level hostnames are used to distinguish. For this to work, a SNI header should be added to the initiator connection (JDK fully supports this since version 7 I think).
https://en.wikipedia.org/wiki/Server_Name_Indication

Describe the solution you'd like
A SNI header should be added to the initiator connection (JDK fully supports this since version 7 I think).

Describe alternatives you've considered
Alternative is to use something like stunnel, or allocate ports per connection, neither of which is nice, given that we have the differentiating mechanism already available.

Additional context
SNI shoul be enabled by default when using TLS -- servers which don't support it simply ignore it.

i have add quickfixj class SessionNotFound but i got error class not found

Describe the bug
A clear and concise description of what the bug is.
I have create java application for fix protocol using quickfixj engine for main class i using this code

package com.dxtr.marketdatarequest.fastmatchmarketdata;

import java.io.FileInputStream;
import java.io.InputStream;
import java.util.UUID;

import quickfix.Initiator;
import quickfix.*;
import quickfix.fix42.*;
import quickfix.LogFactory;
import quickfix.Application;
import quickfix.ConfigError;
import quickfix.DefaultMessageFactory;
import quickfix.FileLogFactory;
import quickfix.FileStoreFactory;
import quickfix.Message.Header;
import quickfix.MessageFactory;
import quickfix.Session;
import quickfix.SessionID;
import quickfix.SessionNotFound;
import quickfix.SessionSettings;
import quickfix.SocketInitiator;
import quickfix.field.BeginString;
import quickfix.field.HeartBtInt;
import quickfix.field.MDEntryType;
import quickfix.field.MDReqID;
import quickfix.field.MarketDepth;
import quickfix.field.MsgType;
import quickfix.field.NoMDEntryTypes;
import quickfix.field.ResetSeqNumFlag;
import quickfix.field.SenderCompID;
import quickfix.field.SubscriptionRequestType;
import quickfix.field.Symbol;
import quickfix.field.TargetCompID;
import quickfix.fix42.Message;

public class FastmatchMDRequest {
    public static void main(String[] args) {
        SocketInitiator socketInitiator = null;
        try {
            SessionSettings sessionSettings = new SessionSettings("/home/ec2-user/fastmatch_server_new/FIXQuote2.cfg");
            Application application = new TestMarketdataRequest();
            FileStoreFactory fileStoreFactory = new FileStoreFactory(sessionSettings);
            FileLogFactory logFactory = new FileLogFactory(sessionSettings);
            MessageFactory messageFactory = new DefaultMessageFactory();
            socketInitiator = new SocketInitiator(application,
                    fileStoreFactory, sessionSettings, logFactory,
                    messageFactory);
            socketInitiator.start();
            SessionID sessionId = socketInitiator.getSessions().get(0);
            sendMarkeDataRequest(sessionId);
            int i = 0;
            do {
                try {
                    Thread.sleep(1000);
                    System.out.println(socketInitiator.isLoggedOn());
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
                i++;
            } while ((!socketInitiator.isLoggedOn()) && (i < 30));
        } catch (ConfigError e) {
            e.printStackTrace();
        } catch (SessionNotFound e) {
            e.printStackTrace();
        } catch (Exception exp) {
            exp.printStackTrace();
        } finally {
            if (socketInitiator != null) {
                socketInitiator.stop(true);
            }
        }
    }
    private static void sendMarkeDataRequest(SessionID sessionId)
            throws SessionNotFound {
    	UUID uuid = UUID.randomUUID();
        String randomUUIDString = uuid.toString();
		Message message = new Message();
		quickfix.fix42.MarketDataRequest.NoMDEntryTypes group =
				  new quickfix.fix42.MarketDataRequest.NoMDEntryTypes();
		quickfix.fix42.MarketDataRequest.NoRelatedSym group1 =
				  new quickfix.fix42.MarketDataRequest.NoRelatedSym();
		MarketDataRequest marketdatarequest = new MarketDataRequest();
		Header header = marketdatarequest.getHeader();
		header.setField(new BeginString("FIX.4.2"));
		  header.setField(new SenderCompID("MDValueTrade2UAT1"));
		  header.setField(new TargetCompID("Fastmatch1"));
		  header.setField(new MsgType("V"));
		  message.setField(new MDReqID(randomUUIDString));
		  message.setField(new SubscriptionRequestType((char) 1));
		  message.setField(new MarketDepth(1));
		  message.setField(new NoMDEntryTypes(1));
		  group.setField(new MDEntryType((char) 1));
		  message.addGroup(group);
		  group1.setField(new Symbol("ALL"));
		  message.addGroup(group1);
		  try
		  {
		  Session.sendToTarget(message);
		  System.out.println("message" + message);
		  }catch (Exception ex)
		  {
			  System.out.println("error" + ex);
			  }
    }
}

To Reproduce
Steps to reproduce the behavior.
Or even better, a unit test or reproducer.
when i run my java application i got an error like

[ec2-user fastmatch_server_new]$ java -jar fastmatchmarketdata-0.0.1-SNAPSHOT.jar 
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.NoClassDefFoundError: quickfix/SessionNotFound
        at java.lang.Class.getDeclaredMethods0(Native Method)
        at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
        at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
        at java.lang.Class.getMethod0(Class.java:3018)
        at java.lang.Class.getMethod(Class.java:1784)
        at sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
        at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
Caused by: java.lang.ClassNotFoundException: quickfix.SessionNotFound
        at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
        ... 7 more

it is like cannot found java class of quickfixj/SessionNotFound
Expected behavior
A clear and concise description of what you expected to happen.

system information:

  • OS: linux
  • Java version JDK8
  • QFJ Version 2.1.1

Additional context
Add any other context about the problem here.

how to fix my problem. i have search using google but i cannot get the fix how to solve it

your help very appreciate

Race caused by static cache in DefaultSessionFactory (QFJ-971)

I think there's a race condition caused by static cache in DefaultSessionFactory, which sometimes effectively corrupts the message.

This can happen when:

  • there are 2 sessions each with it's own IO thread
  • both session use same dictionary file
  • both sessions receive first message of a particular type around the same time

Long story:
We use QuickFIX to stub FIX API-s in acceptance tests. Stub is a separate process, and acceptors are dynamically created and destroyed during tests execution. Each acceptor runs it's own IO thread.
Recently we added support for an API which requires sending same message to 2 endpoints for redundancy purposes. So the test starts 2 sessions with the same dictionary file, and verifies that both received expected message.
However now we occasionally see very strange FieldNotFound exceptions when trying to access a field of repeating group, although raw fix logs clearly shows the tag is present.

After some poking around I think there is a race condition caused by interplay of static cache in DefaultSessionFactory:

private static final SimpleCache<String, DataDictionary> dictionaryCache = new SimpleCache<>(path -> {
        try {
            return new DataDictionary(path);
        } catch (ConfigError e) {
            throw new QFJException(e);
        }
    });

and DataDictionary::getOrderedFields:

    public int[] getOrderedFields() {
        if (orderedFieldsArray == null) {
            orderedFieldsArray = new int[fields.size()];
            int i = 0;
            for (Integer field : fields) {
                orderedFieldsArray[i++] = field;
            }
        }

        return orderedFieldsArray;
    }

My reconstruction of events is this:

  • 2 sessions receive same message; because of the static cache they inadvertently share same instance of DataDictionary
  • when parsing message both threads descend into repeating groups and call DataDictionary::getOrderedFields
  • first thread starts lazy initialisation and updates orderedFieldsArray
  • now second thread may see partially constructed array before first thread finished building it
  • second thread passes this partially constructed array to FieldOrderComparator and now we have effectively mutable comparator; which gives one result before first thread finished building array, and different resut later
  • so if message is parsed before array is built but accessed after, internal structure of a field map is inconsistent with what comparator tells - hence FieldNotFound

The simple fix would be to make getOrderedFields idempotent like that:

    public int[] getOrderedFields() {
        if (orderedFieldsArray == null) {
            orderedFieldsArray = toIntArray(fields); 
        }

        return orderedFieldsArray;
    }
    
    private static int[] toIntArray(Collection<Integer> intCollection)
    {
        int result[] = new int[intCollection.size()];
        int i = 0;
        for (Integer field : intCollection) {
            result[i++] = field;
        }
        return result;
    }

However I wonder if there are other possible dangers of sharing dictionary? We certainly saw cases when dictionary validation properties were accidentally modified because of this static cache (although it was bad session configuration on our part).

QuickFIX/J uses unrecognized xerces property

Describe the bug
A clear and concise description of what the bug is.

Quickfix cannot load configuration file using XML parser from Xerces on runtimes where Xerces would throw an error configuring a property on the xml parser

To Reproduce
Steps to reproduce the behavior.
Or even better, a unit test or reproducer.

Reported at Apache Camel:
https://issues.apache.org/jira/browse/CAMEL-15281

Expected behavior
A clear and concise description of what you expected to happen.

Quickfix should for the following code

    factory.setAttribute(XMLConstants.ACCESS_EXTERNAL_DTD, "");
    factory.setAttribute(XMLConstants.ACCESS_EXTERNAL_SCHEMA, "");

Use a try .. catch and maybe do a WARN log to indicate the attribute was not possible to configure

system information:

  • OS: [e.g. Linux]
  • Java version [e.g. JDK8]
  • QFJ Version [e.g. 2.1.1]

2.2.0

Additional context
Add any other context about the problem here.

Support setting a default UtcTimestampPrecision in the quickfixj-codegenerator Maven plugin

Is your feature request related to a problem? Please describe.
When using the quickfixj-codegenerator Maven plugin to generate QuickFIX/J field classes for a FIX system that uses nano seconds as precision for all UTCTIME, UTCTIMEONLY and UTCTIMESTAMP fields, it would have bee nice to be able to set nano seconds as a default precision for all the generated UTCTIME, UTCTIMEONLY and UTCTIMESTAMP fields.

If this was supported, we could do newOrderSingle.set(TransactTime(transactTime)) instead of newOrderSingle.setUtcTimeStamp(TransactTime.FIELD, transactTime, UtcTimestampPrecision.NANOS).

Describe the solution you'd like
Add a parameter, utcTimestampPrecision, to the quickfixj-codegenerator Maven plugin that takes a value from UtcTimestampPrecision and using that as a default in the classes UtcTimeField, UtcTimeOnlyField, UtcTimeStampField and their generated sub classes.

Describe alternatives you've considered
Using the FieldMap.setUtcTimeStamp method etc. like described above.

Additional context
None.

While looping issue

Hi,
As I know, the messages is processed in the block method. If the application gets unexpected exception(nullpointer exception) or different exception in this method, the "while loop" can break and the messages can not process. I think, the "try catch" blocks should be cover whole "the while loop" block. So that when the application gets unexpected exception, "while loop" won't be broken and the messages is processed. This method can be like below.

private void block() {
	while (true) {
		try {
			synchronized (this) {
				if (isStopped) {
					if (!eventQueue.isEmpty()) {
						final List<SessionMessageEvent> tempList = new ArrayList<>(eventQueue.size());
						queueTracker.drainTo(tempList);
						for (SessionMessageEvent event : tempList) {
							event.processMessage();
						}
					}
					if (stopTime == 0) {
						stopTime = SystemTime.currentTimeMillis();
					}
					if (!sessionConnector.isLoggedOn() || SystemTime.currentTimeMillis() - stopTime > 5000L) {
						sessionConnector.stopSessionTimer();
						// reset the stoptime
						stopTime = 0;
					}
					return;
				}
			}

			SessionMessageEvent event = getMessage();
			if (event != null) {
				event.processMessage();
			}
			//Excepiton types can be seperate.
		} catch (Exception e) {
			Thread.currentThread().interrupt();
		}
	}
}

Rate-limiting for resend request responses

When a session reconnects mid-session it's possible for it to miss a high volume of messages.

At the application level it's possible for the application to self-throttle the release of messages.

It would be useful if resend-request responses could be optionally-throttled to only release messages at a given rate.

This would ideally take the form of a per-session configuration parameter e.g. ResendRequestMaxMessagesPerSecond.

An alternative is to implement it in the form of MINA filters, however this solution can lead to high memory usage because MINA will be forced to store in memory all the resend response messages while they are dequeued.

support of backlog configuration

feature request
for high currency scenario, it always require to upsize applications default backlog

proposed solution
add config support, for example in acceptor.cfg

Additional context
I have checked on my server, the default backlog is 50

We incorrectly check for ApplVerID on admin messages

Describe the bug

We check for ApplVerID even on session level (admin) messages. We only exclude logon from this.
Per the FIXT.1.1 spec session level messages aren't versioned using the ApplVerID. So we shouldn't check them.

We don't even need the ApplVerID for the session level messages as we then use the session dictionary for validating the message.

To Reproduce

Apply MessageUtils.parse on a logout message.

Expected behavior

We should process the message regardless.

Actual behavior*

We throw an InvalidMessage Exception in MessageUtils.getApplVerID

https://github.com/quickfix-j/quickfixj/blob/master/quickfixj-core/src/main/java/quickfix/MessageUtils.java#L136

https://github.com/quickfix-j/quickfixj/blob/master/quickfixj-core/src/main/java/quickfix/MessageUtils.java#L185

system information:

  • OS: Linux
  • Java version: Java 8
  • QFJ Version: 2.2.0-SNAPSHOT

Additional context

This is commonly seen in the real world in the following scenario:

  1. Logon
  2. Logout with no ApplVerID due to bad sequence numbers with no default set for the session
  3. InvalidMessage thrown

A test is needed to defend QuickFIXJ compatibility with FIX Dictionary Files generated from FIX Orchestra

The FIX Orchestra project includes a utility to generate QFJ compatible dictionaries from an Orchestra Repo https://github.com/FIXTradingCommunity/fix-orchestra

The Orchestra model and the QuickFIX model are inconsistent in the way that they deal with Components and Groups.
QuickFIX does not normalise Groups, Groups are defined in situ within the containing component.
There are a small number of cases where Groups are required.
To represent this without requiring that the Component and Group be defined in situ in the message the Group must be defined as required within the Component. e.g. <group name="NoPartyIDs" required="Y">
There are cases such as Component "Parties" where the component is not always required.
QuickFIXJ validation does behave correctly in this case:

  • given that the Component is not required when the contained Group is required and the Component is absent then validation succeeds
  • given that the Component is required when the contained Group is required and the Component is absent then validation fails
  • given that the Component is required when the contained Group is required and the Component is present then validation succeeds

Providing a test case will guard against possible regressions that would introduce incompatibility between FIX Orchestra tooling and QuickFIXJ.

Please see also 👍
FIXTradingCommunity/fix-orchestra#104

There's considerable idle memory churn due to the construction of `Calendar` objects for reset calculations

Describe the bug
An idle QuickFIXJ application produces a stream of garbage to be collected in the form of throwaway (Gregorian)Calendar objects

To Reproduce

  • Create a QFJ application that has a Session attached
  • Observe the memory behaviour and see what objects are being allocated and garbage collected.

Expected behaviour

  • Minimal (ideally 0) use of additional memory without traffic being passed

System information:

  • OS independent
  • Java version 8
  • QFJ Version - 2.2.0-SNAPSHOT

Additional context
The source of this is the DefaultSessionSchedule class which creates (and invokes creation of) Calendar objects every time it's called.

For example, the MessageStore implementations all retain a Calendar object for the creationTime. However the API exposes a getCreationTime() method - and so the store constructs a DateTime from the calendar. But DefaultSessionSchedule uses a Calendar, so it then reconstructs a Calendar object from that date time to then do the comparison.

This all happens every time isSessionTime is called - which (at least) is every time Session.next() is called - which is up to once a second.

UserDefinedFields is only validated on message parsing in a very specific circumstance

Describe the bug

We only check field validity in a specific circumstance

To Reproduce

String ioiWithUserDefinedField = 
    "8=FIX.4.2|9=75|35=6|49=Target|56=Sender|128=BUYSIDE|115=BIG-BANK-PLC|34=1|52=20160531-09:22:01.625|" +
    "23=001|28=N|55=VOD.C|62=20170531-09:22:01.625|130=N|15=GBP|44=13.37|54=1|58=This is a comment|27=122|22=1|48=BH4HKS3|5001=ABC|10=206|";
String ioiInFixFormat = ioiWithUserDefinedField.replaceAll("\\|","\u0001")
DataDictionary fix42Dictionary = new DataDictionary("FIX42.xml");
fix42Dictionary.setCheckUserDefinedFields(true)
Message message = new Message(ioiInFixFormat, fix42Dictionary, true);

Expected behavior
Constructor should throw FieldException with message: "Invalid tag number, field=5001"

system information:

  • Linux
  • Java version: JDK 8
  • QFJ Version: 2.2.0-SNAPSHOT

Additional context

This may be by design but it's not clear that it'll happen from the API. The boolean you pass is called validate, it's reasonable to expect that it should validate according to the dictionary settings.

The only circumstance we check for user defined fields is in parseGroup:

if (checkFieldValidation(parent, parentDD, field, msgType, doValidation, group)) {

Support for FIX Latest

It would be good for QFJ to support FIX Latest from FTC. If this is done by means of a dependency on the published FIX Latest Repository (Orchestration).

I would suggest that FIXLatest be added as an additional module to preserve backward compatibility of the existing FIX50sp2 module.

This would also provide a ready means for users to depend on their customized repositories instead of the base-line FIX Standard.

The FIX Latest repo includes some extension packs that break the QFJ build so code generated for messages in the new FIX50sp2Latest needs to be slightly different. The problem is where components and fields have the same name. As Java does not overload based on return type the method name for components must change. For backward compatibility this should not be applied to messages for prior versions of the standard.

I will provide an example build. See the branch here : https://github.com/david-gibbs-ig/quickfixj/tree/fix-orchestra
This above is a work in progress,

`quickfix.ApplicationExtended#onBeforeSessionReset(SessionID)` called only if session has responder and is logged on

Describe the bug
quickfix.ApplicationExtended#onBeforeSessionReset(SessionID) is called only if the session has a responder and is logged on.

To Reproduce
Setup an application with StartTime and EndTime and wait for quickfix.Session#reset() to get called by quickfix.Session#resetIfSessionNotCurrent(SessionID, long) before logging on when entering StartTime with a non current session.

Expected behavior
quickfix.ApplicationExtended#onBeforeSessionReset(SessionID) gets called even if the session is not logged on. Proposed solution: Move the call to quickfix.ApplicationExtended#onBeforeSessionReset(SessionID) to the top of the try block.

system information:

  • OS: CentOS
  • Java version: JDK8
  • QFJ Version: 2.2.0

Additional context
Not sure if this qualifies as a bug or if it actually works as intended. Please discard and close the issue if it works as intended.

How to handle NewOrderList in an ExecutionReport

Hi i am new to fix protocols, i was wondering how to process a NewOrderList from an ExecutionReport. As per a fix message document on the internet https://gainfutures.com/api/html/133e2886-5411-44b7-b781-2311a45ab540.htm i came to know that every order in the NewOrderList will be processed one at a time in the ExecutionReport.

From the above doc:
order statuses and trades are reported for individual orders, not for the list as whole.

Is there a way i can get the list of order details back from the ExecutionReport event listener using below

public void onMessage(ExecutionReport report, SessionID sessionID) {
        // handler implementation
}

Connecting over an authenticating HTTP proxy fails

We are trying to connect over an HTTP proxy that enforces authentication, i.e. user and password need to be set. But even though we put them into the dictionary (and set the appropriate proxy type and the correct proxy host and port) the connection fails.

Snooping into the network traffic with wireshark reveals that, at first, a CONNECT request is sent by our application to the proxy. As usual, this initial request contains no authentication data, and is answered by the proxy with an 407 Authentication Required. The expected behaviour would be that a new CONNECT request is send, this time with a Basic Proxy-Authorization header containing the user and password. Instead, the connection is closed.

I ran the code in a debugger and found that the problem seems to be that in quickfix.mina.ProtocolFactory (line 145 to 150) a prefered order of the authentication methods is set into the proxyIoSession. This order, however, has NO_AUTH at the first position. So that, after receiving the 407 and when going through the possible authentication methods, Mina's HttpSmartProxyHandler first encounters a NO_AUTH and sees this as a sign that every further search for an authentication method is futile. And therefore closes the connection.

The prefered order of the authentication methods in ProtocolFactory seems to be rather obviously wrong. But instead of correcting it, you might want to just not set it, in which case Mina would go with their own idea of which method to prefer, which seems to be a good idea.

I forced the prefered order list to be null in my debugger, and was able to succesfully connect through the proxy.

QFJ does not wait for Logout reply when EndTime is reached

Good afternoon,

We are using QuickFix 2.2.0 and FIX4.2 messages version.

We configured our session.cnf with StartTime and EndTime. Our problem is when time is equal to EndTime, QuickFix sends logout message but doesn't wait to response and generates a reset event: Disconnecting: Session reset.

Could you help us?

Thank you, regards.

Connection timeout on one Session will block timer-related tasks on other Sessions (cf. QFJ-291)

On some of our FIX nodes we noticed that sometimes Heartbeats weren't sent out when they were due. After analysis and taking some stack dumps we noticed that QFJ was trying to connect to a FIX session at that time. But the connection attempt was not refused but timed out. This will block the thread (running on a SingleThreadedExecutor) for a maximum of 2 seconds.

private void pollConnectFuture() {
try {
connectFuture.awaitUninterruptibly(CONNECT_POLL_TIMEOUT);

There already was a JIRA issue for this (https://www.quickfixj.org/jira/browse/QFJ-291) which introduced the 2 second timeout. Before it probably waited until the network timeout triggered.

The same SingleThreadedExecutor is also responsible to call the Session.next() method (which among other things is responsible to generate Heartbeats) for all Sessions.

As a first step we will introduce a separate Executor with a few threads that will deal with the Initiator connection attempts. This will

  • enable concurrent establishment of Sessions even if one or two are timing out when trying to connect.
  • not block calling Session.next() for other Sessions.
    (Later we might also upgrade the SingleThreadedExecutor to have more threads. But we need to take care that Session.next() gets called always by the same thread from the pool to prevent possible concurrency/visibility issues)

The latter point is also tracked in https://www.quickfixj.org/jira/browse/QFJ-555.

Will open a PR for this shortly. Just opening this issue so I don't forget about it.

TimestampPrecision config can not work

Describe the bug
I found default TimestampPrecision is set to 3,but when i change it to 6, it can not work.

To Reproduce
[DEFAULT]
TimestampPrecision=6

Expected behavior
�52=20210110-01:57:10.176 can change to �52=20210110-01:57:10.176123

system information:

  • OS: macos
  • Java version JDK11
  • QFJ Version [2.2.0]

Additional context
none

Certain JMX beans' attributes are broken due to a bug in CompositeTypeFactory

Describe the bug
Certain attributes are unavailable via JMX.

To Reproduce
Connect via JMX and check the attributes.

Expected behavior
Composite values should be visible via JMX.

Additional context
org.quickfixj.jmx.openmbean.CompositeTypeFactory allows to add items without description via #defineItem(java.lang.String, javax.management.openmbean.OpenType) - which is actually being used . However, #205 PR has mistaken actual functionality for a typo.

Currently broken items seem to be:

  • AcceptorAddresses
  • Endpoints
  • Sessions/Session
  • Sessions/SessionID

I will be producing fix soon.

image

NullPointerException during quickfix initialization

I've met with NPE inside quickfix on initialization.
I've seen it in v2.0.0, but apt code in master is still same:

Thread.currentThread().getContextClassLoader().loadClass(factoryClassName);

Root cause: Thread.getContextClassLoader() is not guaranteed to be not-null according to spec

@return the context ClassLoader for this Thread, or {@code null} indicating the system class loader (or, failing that, the bootstrap class loader)

In my case it so happens first quickfix call was from network library handler, which is done from network library thread, and such a thread missed context class loader.

(I've also noted factoryClass is not assigned in the catch branch -- i.e. even if context classLoader is not null, factory class will be loaded, but not used!)

I don't understand code deep enough to decide what to do with such a case, but generic NPE is definitely not user-friendly. IMHO:

  1. If there is no way to continue without classloader -> it is at least worth to throw ClassNotFoundException with reasonable error description
  2. If it is OK to not load specific factory -> than maybe just ignore that factory loading. But it seems to me in such a case none of factories would be loaded (because class loader would be null during whole initialization), without any hint about the reason -- and this could be even more confusing for user to debug.

Allow custom DataDictionaryProvider in DefaultSessionFactory

Is your feature request related to a problem? Please describe.
Right now, the dictionary has to be read from the file. Specifically, it cannot be read from the classpath (i.e. embedded within a jar).

Describe the solution you'd like
Add another constructor to DefaultSessionFactory which accepts a custom DataDictionaryProvider to be used in create instead of DefaultDataDictionaryProvider.

Describe alternatives you've considered
Alternatives would be adding a Supplier or something along those lines.

Additional context
N/A

Property 'http://javax.xml.XMLConstants/property/accessExternalDTD' is not recognized.

Describe the bug
After upgrading quickfixj to latest version (2.2.0) the code that used to run correctly now throws the following exception:
Caused by: java.lang.IllegalArgumentException: Property 'http://javax.xml.XMLConstants/property/accessExternalDTD' is not recognized. at org.apache.xerces.jaxp.DocumentBuilderFactoryImpl.setAttribute(Unknown Source) at quickfix.DataDictionary.load(DataDictionary.java:883) at quickfix.DataDictionary.read(DataDictionary.java:869)

Expected behavior
The same code should still be running after library update.

system information:

  • OS:Linux, Ubuntu 18.04 LTS
  • Java version JDK8
  • QFJ Version 2.2.0
  • Xerces version 2.12.0

Additional context
This is a piece of software that used to run correctly before, the only change we did was to upgrade the QFJ library. The application embeds QFJ functionality and also requires Xerces for additional functionalities, so we cannot get rid of Xerces or use an older version.

Security Contact Requested

@n33dle and I found a security vulnerability within the quickfixj project. We were hoping to get a core maintainer's contact details to responsibly disclose this vulnerability.

Message processing after stopping the stream

I apologize for my English, translate using Google Translate.

In general, the situation is as follows:
In the Session.next method, each message is checked using the Session.verify.
When the thread is started, the checks pass as expected. But, if for some reason the connection is interrupted, the processing of messages goes from the internal queue of messages that have not yet been processed (for example, this is shown in a private method SingleThreadedEventHandlingStrategy.block -> SessionMessageEvent event.processMessage -> Session.next(message). -> Session.verify). Inside the Session.verify method there is a check implemented as a private method Session.validLogonState, and when the thread is stopped, messages from the internal queue do not pass the check (the Session.verify throws a SessionException ("Logon state is not valid for message (MsgType ="+ msgType + ")").
As a result, if a sufficient number of messages has accumulated in the queue, these messages automatically (except for possible Logout messages) are considered incorrect and go to the logs.

My question is: is this behavior correct when the message queue, after stopping the flow, continues to process messages with unsuccessful results? Or is it possible to implement queue clearing in case of loss of communication or skipping messages that do not fit the condition?

An example of a situation might be the following:
If the message does not pass the time check when the stream is turned on (Session.isGoodTime method), the stream is automatically turned off. But, a Logout message may not yet be sent due to the presence of other messages in the queue, and the stream is already disabled (methods is Session.doBadTime -> Session.logoutWithErrorMessage -> Session.disconnect). As a result, the rest of the queue messages will not be checked by Session.verify.

ThreadedSocketInitiator reloads all sessions if one session fails

Starting with version 2.1.1, I started using the ThreadedSocketInitiator. And there is a problem when, in the event of a failure in one session (for example, the message does not pass the time correctness check - Session.isGoodTime method), all sessions reboot, which takes several seconds. Is it possible to handle reloading only an erroneous session, leaving the remaining sessions in operation?

QFJ upgrade issue from 2.1.1 to 2.2.0 due to XMLConstants.ACCESS_EXTERNAL_DTD

Describe the bug
I am trying to upgrade the QFJ version 2.2.0 from 2.1.1 facing below issue.

java.lang.IllegalArgumentException: Property 'http://javax.xml.XMLConstants/property/accessExternalDTD' is not recognized.
at org.apache.xerces.jaxp.DocumentBuilderFactoryImpl.setAttribute(Unknown Source)
I have analysed this is because of QFJ DataDictionary.java's load method using below code.

private void load(InputStream inputStream) throws ConfigError {
final DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
factory.setAttribute(XMLConstants.ACCESS_EXTERNAL_DTD, "");
factory.setAttribute(XMLConstants.ACCESS_EXTERNAL_SCHEMA, "");
...}
Reason of error is not clearly understood by me(please try to explain) but found that Xerces, xercesImpl jar causing this error.

we are using these jar in some our project code that is why it is needed.

Can anyone try to explain why we have placed these lines? and we are placing then it should be under try and catch block. Is there any way to fix it. Without doing source code changes provide by QFJ. Answer from QFJ team really appreciated.

To Reproduce
Just add Xerces, xercesImpl jar in class path.
Expected behaviour
Does not throw a runtime exception. A normal warning log message is fine.

system information:

  • OS: [e.g. Windpw]
  • Java version [e.g. JDK8]
  • QFJ Version [e.g. 2.2.0]

Pluggable transport

Is your feature request related to a problem? Please describe.
Quite a long time ago, I've been working to make the transport pluggable and to provide one based on Netty. The code is still available here but as I was experimenting against a number of things, it contains more that just the pluggable transport work.

Describe the solution you'd like
If it does make sense I can work on the pluggable transport layer over the following months.

how to reload order data when i restart server?

I found this url https://www.quickfixj.org/usermanual/2.1.0/usage/acceptor_failover.html, I create config file like this :

[default]
FileStorePath=target/data/ordermatch
DataDictionary=FIX42.xml
SocketAcceptPort=8323
BeginString=FIX.4.2
; for python used
TimeStampPrecision=MICROS
; for n2n used
; TimeStampPrecision=MILLIS
SenderCompID=FEME
ConnectionType=acceptor
StartTime=00:00:00
EndTime=00:00:00
FileLogPath=logs
AllowUnknownMsgFields=Y
ValidateUserDefinedFields=N
ValidateIncomingMessage=N
ValidateSequenceNumbers=N
RefreshMessageStoreAtLogon=Y

I try to add a new order, but when i restart server i found order data can not reload from persist data! How can i do in this tiem? thanks!

Setting RefreshOnLogon should also work for Initiators

Up to now, the setting RefreshOnLogon will only refresh the message store for Acceptors.

The .NET and C++ implementations do not have this restriction.

Use case: when adding a dynamic session and changing the sequence number via JMX this apparently works but when Logon is initiated it is done with sequence number 1 again.

NB: probably documentation needs to be adapted also.

Correction in README toadmin

Can I send a PR making a small correction in README.md on the toadmin item?

If you think it's not worth the work I can close this issue, not a big deal!

From

This allows you to add fields before an adminstrative message before it is sent out.

To

This allows you to add fields to an administrative message before it is sent out.

Fail over retry attempt

Hi All,

I'm really new to this community and this is my 1st comment. We are using QuickFix library for our application. There is a requirement received to handle the fail-over scenario for initiator session as below.

  1. If a disconnection experienced for the primary host(SocketAcceptHost), We should try the same connection for a configurable times.

  2. If all the retry attempts are failed only we need to start trying to the other hosts(SocketAcceptHost1, SocketAcceptHost2...SocketAcceptHost[N]).
    As of my understanding, the above requirement is not available with existing code and it seems doable while introducing new parameter to the class "IoSessionInitiator". Please be kind enough to advise me further on this.

As i mentioned, I'm really new to this community and i'm not sure this is the right place to discuss this. If it is not please someone point me to the right direction. Thanks in advance.

LogonTime and LogoutTime

Good afternoon,
we are deploying an application using Quickfix 2.1.1 and FIX4.2 version for messages.

Our issue: Quickfix ignores both parameters: LogonTime and LogoutTime. It's only using StartTime and EndTime.

It only does logout sending logout message but it doesn't wait to logout response.
Could give us some information?

Thank you, regards.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.