A monadic library to resolve object relations with the aim of avoiding the N+1 query problem. The solution requires only pure computations in Scala depending on neither database implementations nor any other frameworks.
Add dependency in your build.sbt
as the following.
libraryDependencies ++= Seq(
"com.github.tarao" %% "bullet" % "0.0.2"
)
The library is available on Maven Central. Currently, supported Scala version is 2.11.
Assume that you have Car
and Engine
classes and, if Car
has an
Engine
, it is resolved by a type class method toEngine
.
type CarId = Long
type EngineId = Long
case class Car(id: CarId)
case class Engine(id: EngineId, carId: CarId)
implicit class CarRelation(val car: Car) extends AnyVal {
def toEngine: Option[Engine] = ...
}
It is quite usual to implement toEngine
method by using a repository
which issues a DB query.
implicit class CarRelation(val car: Car) extends AnyVal {
def toEngine: Option[Engine] = EngineRepository.findByCarId(car.id)
}
val db = ...
object EngineRepository {
def findByCarId(carId: CarId): Option[Engine] = db.run {
sql"SELECT * FROM engine WHERE car_id = $carId LIMIT 1".as[Engine]
}.headOption
}
There is no problem when you resolve an Engine
from a Car
. In
this case, a SELECT
query is executed internally.
val car: Car = Car(1234L)
val engine: Option[Engine] = car.toEngine
// SELECT * FROM engine WHERE car_id = 1234 LIMIT 1
If you have multiple Car
s and want to get their Engine
s, you may
want to write like this.
val cars: Seq[Car] = Seq(Car(1L), Car(2L), Car(3L), ...)
val engines: Seq[Engine] = cars.map(_.toEngine).flatten
// SELECT * FROM engine WHERE car_id = 1 LIMIT 1
// SELECT * FROM engine WHERE car_id = 2 LIMIT 1
// SELECT * FROM engine WHERE car_id = 3 LIMIT 1
// ...
Yes, it works. But there is a problem that the SELECT
query is
executed for each id
of the element of cars
. When you have
hundreds or thousands of cars
, it is likely to be a perfomance
issue.
One way to solve this problem is to JOIN
tables. When you
instantiate Car
s from car
table in your DB, engine
table might
also be INNER JOIN
ed. This is a quite common solution but not the
best one. If you have for example Wheel
s, a Bumper
, and Door
s
for a Car
, you will soon need to JOIN
them all but not all of them
are needed every time. You will have to write instantiation methods
with ugly JOIN
queries for each combination of parts that you need.
Ideally, it would be nice if the last expression executes a single
SELECT
query.
val engines: Seq[Engine] = cars.map(_.toEngine).flatten
// SELECT * FROM engine WHERE car_id IN (1, 2, 3, ...)
Is this possible? In Scala, yes, it is.
All you have to do is to replace toEngine
method to return an
instance of a monad created by HasA.Monadic
with an instance of
HasA[Car, Engine]
, which describes how to resolve Engine
s from
Car
s.
import com.github.tarao.bullet.HasA
implicit class CarRelation(val car: Car) extends AnyVal {
def toEngine = HasA.Monadic(car, hasEngine)
}
val hasEngine: HasA[Car, Engine] = ...
The usage of toEngine
is quite the same except that (1) you have to
write a type of return value (Option[Engine]
or Seq[Engine]
in
this case), (2) you don't need to flatten
anymore.
import com.github.tarao.bullet.Implicits._
val car: Car = ...
val engine: Option[Engine] = car.toEngine
val cars: Seq[Car] = ...
val engines: Seq[Engine] = cars.map(_.toEngine)
The implementation of hasEngine
should resolve Engine
s from Car
s
in a single query. HasA[Car, Engine]
has an interface for that
named map()
, whose type is Seq[Car] => Seq[Engine]
. Then the
implementation whould be the following.
val hasEngine: HasA[Car, Engine] = new HasA[Car, Engine] {
def map(cars: Seq[Car]): Seq[Engine] = db.run {
sql"SELECT * FROM engine WHERE car_id IN (${cars.map(_.id)})".as[Engine]
}
}
Note that map()
method is used for resolving both Option[Engine]
and Seq[Engine]
. In our example, toEngine
results in executing a
SELECT
-WHERE
-IN
query in the both cases.
val car: Car = Car(1234L)
val engine: Option[Engine] = car.toEngine
// SELECT * FROM engine WHERE car_id IN (1234L)
val cars: Seq[Car] = Seq(Car(1L), Car(2L), Car(3L), ...)
val engines: Seq[Engine] = cars.map(_.toEngine)
// SELECT * FROM engine WHERE car_id IN (1, 2, 3, ...)
The key mechanism is a monad, which is a return value of toEngine
.
In the last example, we receive a monad as a variable of type
Option[Engine]
or Seq[Engine]
. This is actually an implicit
conversion. If we make it explicit, the example looks like this.
val car: Car = ...
val engine: Option[Engine] = car.toEngine.run
val cars: Seq[Car] = ...
val engines: Seq[Engine] = cars.map(_.toEngine).run
It is run()
which actually calls HasA[].map()
. Until then, the
invocation of HasA[].map()
is postponed inside the monad. If the
receiver of run()
is a list of monads, it will organize them into an
argument of a single invocation of HasA[].map()
. (You may wonder
how it is possible since each monad value has its own instance of
HasA[]
. It is actually only the first one in the list to be used.)
The above story does not describe the way using our monad as a monad. Actually, it is not necessarily a monad as long as it is some kind of a deferred object. It is a monad just for convenience.
Let's see an example. Suppose that an Engine
has its Crankshaft
and there is a type class method toCrankshaft
defined in the same
way as toEngine
. If we don't have the monadic feature, we need to
look up a Crankshaft
of a Car
via an Engine
in the way like
this.
val car: Car = ...
val engine: Option[Engine] = car.toEngine
val crankshaft: Option[Crankshaft] =
engine.map(_.toCrankshaft: Option[Crankshaft]).flatten
If we use the monadic feature, it can be written like this.
val car: Car = ...
val crankshaft: Option[Crankshaft] = for {
e <- car.toEngine
c <- e.toCrankshaft
} yield(c)
This is much easier to read especially when you need a complex
operation on e
and/or c
.
As you have seen in the overview, only things you have to do are
to implement HasA[].map()
and to provide a monad factory using it.
- Extend
HasA[From, To]
and implementmap: Seq[From] => Seq[To]
. - Provide a monad factory which returns
HasA.Monadic(from, hasA)
where:from
is an instance ofFrom
hasA
is an instance ofHasA[From, To]
You can specify a list type as To
type argument of HasA[From, To]
.
In this case, you can resolve a single value as a Seq[_]
instead of
Option[Seq[_]]
, or multiple values as a Seq[_]
instead of
Seq[Seq[_]]
. For example, if you have HasA[Car, Seq[Wheel]]
and
toWheels
returns a monad, then these can be used as the following.
val car: Car = ...
val wheels: Seq[Wheel] = car.toWheels
val cars: Seq[Car] = ...
val totalWheels: Seq[Wheel] = cars.map(_.toWheels)
- The same as
HasA[From, To]
butTo
is a list type - The result can be flattened automatically
Sometimes you may want to merge related two objects into one. For
example, if you have Car
s and their Engine
s, you may want to have
values of CarWithEngine
s where those are merged. To do this, you
can use another interface Join.Monadic
to define a type class method
withEngine
.
import com.github.tarao.bullet.Join
type CarWithEngine = (Car, Engine)
implicit class CarRelation(val car: Car) extends AnyVal {
def withEngine = Join.Monadic(car, joinEngine)
}
type JoinEngineToCar = Join[CarWithEngine, CarId, Car, Engine]
val joinEngine: JoinEngineToCar = new JoinEngineToCar {
def map(cars: Seq[Car]): Seq[Engine] = ... // the same as HasA[Car, Engine]
def leftKey(car: Car): CarId = car.id
def rightKey(engine: Engine): CarId = engine.carId
def merge(car: Car, engine: Engine): CarWithEngine = (car, engine)
}
This time you have four methods to implement. map()
, leftKey()
,
rightKey()
and merge()
. map()
is the same as in
HasA[Car, Engine]
. leftKey()
and rightKey()
provides how you
associate one object to another. In this case, a Car
and its
Engine
should share their CarId
. associated objects are passed to
merge()
.
The usage is quite similar to toEngine
.
val car: Car = ...
val enginedCar: Option[CarWithEngine] = car.withEngine
val cars: Seq[Car] = ...
val enginedCars: Seq[CarWithEngine] = cars.map(_.withEngine)
- Extend
Join[Result, Key, Left, Right]
and implement four methods:map: Seq[Left] => Seq[Right]
leftKey: Left => Key
rightKey: Right => Key
merge: (Left, Right) => Result
- Provide a monad factory which returns
Join.Monadic(left, join)
where:left
is an instance ofLeft
join
is an instance ofJoin[Result, Key, Left, Right]
An Option[]
return value of run()
may be None
if HasA[].map()
returns an empty list. In this case, you can provide a default value
to ensure having some value returned. This is done by providing an
implicit value of Monad.Default[]
. If you provide a default value,
the return value can be received without being wrapped by Option[]
.
For example, the following code defines a default value for an
Engine
.
implicit val defaultEngine: Monad.Default[Engine] =
Monad.Default[Engine](Engine(0L, 0L))
val car: Car = ...
val engine: Engine = car.toEngine
In this case, note that the implicit value must be visible in the
scope where the invocation of run()
or the implicit conversion
occurs.
For Join[]
, you should be careful that you have two choices of types
to provide a default value, either Result
or Right
of
Join[Result, Key, Left, Right]
. If you provide a default value for
Result
, then you will always get a value by run()
on a single
monad but still get some values lacked by run()
on multiple monads.
You should provide a default value for Right
to avoid this. In this
time, the implicit value must be visible in the scope where the
invoction of Join.Monadic()
occurs.
You may think that resolving object relations on an implicit
conversion is too aggressive. There is a way not to allow implicit
conversions and force explicit run()
s instead. Only you have to do
is not to import com.github.bullet.Implicits._
. Other things will
work fine without this.
- Copyright (C) INA Lintaro
- MIT License