jpzk / kafcache   1.3.0

Apache License 2.0 GitHub

Kafka Streams + Memcached (e.g. AWS ElasticCache) for low-latency in-memory lookups

Scala versions: 2.12

kafcache

Codacy Badge Maven Central

In-memory Kafka Streams state store backends for low latency state store lookups. In the current version only memcached is available.

Dependency

libraryDependencies += "com.madewithtea" %% "kafcache" % "1.3.0"

Use Memcached

Memcached does not support binary keys, therefore the byte arraywill be serialized as hex string. For more information on Memcached have a look at its documentation on https://memcached.org/.

  import com.madewithtea.kafcache.MemcachedStoreSupplier

  val store = Stores
    .keyValueStoreBuilder(
      new MemcachedStoreSupplier("state-store-name", "localhost:11211"),
      Serdes.ByteArray(),
      Serdes.ByteArray()
    )
  .withLoggingEnabled(new java.util.HashMap[String, String]())

Skip recovery

The following code will initialize a state store but it will not write to it in the recovery process. Hence, the recovery is much faster than doing actual writes.

  import com.madewithtea.kafcache.MemcachedStoreSupplier

  val store = Stores
    .keyValueStoreBuilder(
      new MemcachedStoreSupplier("state-store-name", "localhost:11211", recover = false),
      Serdes.ByteArray(),
      Serdes.ByteArray()
    )
  .withLoggingEnabled(new java.util.HashMap[String, String]())

Statistics

5 Dependencies

No Dependent