Giter VIP home page Giter VIP logo

twarc's Introduction

twarc

Travis status

A Simple Clojure wrapper around Quartz Scheduler.

Artifacts

Clojars Project

Why?

There are a few Clojure libraries for working with Quartz, but each one has a fatal flaw (at least one):

  • Quartzite
    • Global state, thread bindings
    • You can't explicitly pass some type of state (e.g., a db connection) to jobs
    • Complex API
  • Immutant Scheduling
    • You can't explicity pass some type of state (e.g., a db connection) to jobs
    • You can't use it with persistent JobStores and in Clustered environments
    • Complex dependencies

The main disadvantage of these libs is that you can't use it in the right way, i.e. without any global state but with strong dependency management (see stuartsierra/component)

Features

  • No global state
  • You can pass any instance-aware context to jobs
  • Data structure-centric API (see examples)
  • Jobs are usual vars with function (i.e. defn)
  • Stateful jobs
  • No magic
  • stuartsierra.component support out of the box
  • Quartz' Listeners support via core.async channels

Usage

Basic config (see Quartz Configuration Reference):

(require '[twarc.core :as twarc])

(def props {:threadPool.class "org.quartz.simpl.SimpleThreadPool"
            :threadPool.threadCount 1
            :plugin.triggHistory.class "org.quartz.plugins.history.LoggingTriggerHistoryPlugin"
            :plugin.jobHistory.class "org.quartz.plugins.history.LoggingJobHistoryPlugin"})

;; Scheduler supports component/Lifecycle protocol and clojure.lang.Associative (its
;; Clojure record), so you can simply drop it into your system map. Or use some other DI
;; system.

(def sched (-> (twarc/make-scheduler props) (twarc/start)))

defjob macro defines two functions, in this case test-job and test-job*. test-job* is an actual job with the body provided by you. It executes in Quartz' thread pool. Generated test-job is a helper function that can be used for scheduling jobs.

Job function accepts scheduler instance as the first argument, and the rest of the arguments are passed on to job scheduling.

(twarc/defjob test-job
  [scheduler name message]
  (prn "Message for!" name message))

Let's run it!

;; If you use cider, note that Quartz threads know nothing about repl's stdout. So keep an eye on
;; messages in nrepl-server buffer

(test-job sched ["Andrew" "Hello world"])

That's all. The first argument is a scheduler instance; the second one is a vector of arguments, and optional tail arguments are options for schedule-job function (job and trigger params actually – see Quartz documentation for details).

You can schedule execution of any defn without a helper:

(defn test-job2
  [scheduler name message]
  (prn "Message  from!" name message))

(twarc/schedule-job sched #'test-job2 ["Petr" "Hi world!"])

Define simple or cron trigger via map:

(test-job sched ["Andrew" "Hello world"] :trigger {:simple {:repeat 5 :interval 1000}})

(test-job sched ["Andrew" "Hello world"]
          :job {:identity "eternal job"}
          :trigger {:cron "*/10 * * * * ?"})

(twarc/delete-job sched "eternal job")

Persistent JobStore

You can persist your jobs and triggers in JDBC-store.

First of all, you need to create tables, see these scripts – https://github.com/quartz-scheduler/quartz/tree/v2.3.2/quartz-core/src/main/resources/org/quartz/impl/jdbcjobstore

Secondly, configure Quartz for your store. You should also pick a well-defined name for your scheduler:

(def persistent-props
  (assoc props
    :jobStore.class "org.quartz.impl.jdbcjobstore.JobStoreTX"
    :jobStore.driverDelegateClass "org.quartz.impl.jdbcjobstore.PostgreSQLDelegate"
    :jobStore.tablePrefix "QRTZ_"
    :jobStore.dataSource "db"
    :dataSource.db.driver "org.postgresql.Driver"
    :dataSource.db.URL "jdbc:postgresql://localhost:5432/db_name"
    :dataSource.db.user "user"
    :dataSource.db.password "pass"))

(def persistent-sched (-> (twarc/make-scheduler persistent-props {:name "main-sched"})
                          (twarc/start)))

In this example we can also see how to configure a job and a trigger. With the :state param provided, a job becomes a Stateful job, and the job function accepts state as the second argument and should return updated state.

(twarc/defjob test-statefull-job
  [scheduler state i]
  (prn "State!" state)
  (update-in state [:counter] + i))

(test-statefull-job persistent-sched [4]
                    :job {:state {:counter 1}}
                    :trigger {:simple {:repeat :inf :interval 1000}})

And now stop and start a new scheduler without scheduling a task. Our previously scheduled task will continue executing.

(twarc/stop persistent-sched)
(def persistent-sched2 (-> (twarc/make-scheduler persistent-props {:name "main-sched"})
                           (twarc/start)))

Listeners

You can define listeners of some events with core.async channels.

(require '[clojure.core.async :as a])
(def executed (twarc/add-listener persistent-sched2 {:everything true} :was-executed))

(loop []
  (prn "--EXECUTED!" (->  (a/<!! executed) .getJobDetail .getJobDataMap (get "state")))
  (recur))

License

Copyright © 2015 Andrew Rudenko

Distributed under the Eclipse Public License either version 1.0 or (at your option) any later version.

twarc's People

Contributors

0xtmphey avatar cpmcdaniel avatar prepor avatar tangrammer avatar therabidbanana avatar webmonarch avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

twarc's Issues

Job :recovery option doesn't appear to call the right function

I've been trying to figure out how to gracefully handle restarts with twarc as my job scheduler and found documentation about the requestsRecovery option, which I would think is handled by {:job {:recovery true}} - it appears the naming has changed though - as I get a runtime error:

java.lang.IllegalArgumentException: No matching field found: withRecovery for class org.quartz.JobBuilder

It looks like the option should call (.requestRecovery true) instead?

http://www.quartz-scheduler.org/api/2.2.1/org/quartz/JobBuilder.html#requestRecovery()

Support a manually created trigger

There are lots of kinds of trigger that twarc doesn't have a data mapping for. If it skipped over converting the trigger if it already "looked" like a trigger, then this would work.

Stateful jobs don't persist changes in state

If I upgrade from 0.1.10 to 0.1.12 or later, stateful jobs that are backed by a database stop working.

This is caused by this change:
https://github.com/prepor/twarc/pull/7/files#diff-a72d3f0df7c6d43e920442fdc42b450aR25

After that change you are updating the merged map here:
https://github.com/prepor/twarc/pull/7/files#diff-a72d3f0df7c6d43e920442fdc42b450aR29

Updating the merged map does nothing to the original JobDataMap so any changes are not persisted at all.

You can reproduce this by running the stateful job example from the readme, the counter will not increase at all.

The solution is to change line at https://github.com/prepor/twarc/blob/master/java/twarc/TwarcStatefullJob.java#L29

To context.getJobDetail().getJobDataMap().put("state", result);

Return job from schedule-job?

Is there anything wrong with returning the job (JobDetail instance) from the schedule-job function? I'm wondering if the JobKey would be set on it at that point, or if I would need to use a Listener to capture that.

quartz 2.3.0

There is any well known blocker for updating quartz to newest version? I can start working on it because in my current project we plan to use twarc because is quite simple to use and source code is not so complicated compared to other quartz wrappers.

Incorrect docstring on `matcher`

Matcher's docstring claims that {:everything true} is the syntax for an everything matcher, but this seems not to be the case. Instead the syntax is :everything:

user=> (require '[twarc.core :as twarc])
nil
user=> (twarc/matcher {:everything true} :job)
nil
user=> (twarc/matcher :everything :job)
#object[org.quartz.impl.matchers.EverythingMatcher 0x5b7ff8cf "org.quartz.impl.matchers.EverythingMatcher@bb034737"]

May I suggest that the fix leaves the old behavior in order for anyone who has worked around it to have their code continue to work?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.