Extensible Free Monad Effects

https://blog.oyanglul.us/scala/3-layer-cake

Do one thing and do it well micro birds library series

libraryDependencies += "us.oyanglul" %% "luci" % <version>"

Table of Contents

• 鸕鶿lu ci
  • The Problem
  • The Ultimate Solution
    • Some Effects out of the box
    • Step 1: Create DSL
    • Step 2: Compile the Program
    • Step 3: Run Program
  • Create Your Own Effect
    • Step 1: Create Data Type
    • Step 2: Create Compiler
    • Step 3 Use the effect

When you want to mix in some native effects into your Free Monad DSL, some effects won't work

for instance, we have two effects IO and StateT, and we would like to do some math using StateT

Here is the Program

import Free.{liftInject => free}
type Program[A] = EitherK[IO, StateT[IO, Int, ?], A]
type ProgramF[A] = Free[Program, A]

def program : Program[Int] = for {
  initState <- free[Program](StateT.get[IO, Int])
  _ <- free[Program](IO(println(s"init state is $initState")))
  _ <- free[Program](StateT.modify[IO, Int](_ + 1))
  res <- free[Program](StateT.modify[IO, Int](_ + 1))
} yield res

and Interpreters

def ioInterp = FunctionK.id[IO]
def stateTInterp(initState: Int) = Lambda[StateT[IO, Int, ?] ~> IO[(Int, ?)]] { _.run(initState)}

If we run the program

program foldMap (ioInterp or stateTInterp(0))

Guess what, it doesn't work, the result will be 1 not 2

because we run the state for each effect separately in the stateTInterp

One of the option is to use FreeT

But with FreeT:

• you can only mixin 1 effect, what if I have multiple effects that I want them to be stateful across the whole program.
• all other effects need to be lift to FreeT as well. It might have huge impact to our existing code base that is Free already.

is using both meow-mtl and ReaderT/Kleisli, then we can easily integrate mtl into Free Monad Effects

1. instead of using interpreter Program ~> IO, we can use Program ~> Kleisli[IO, ProgramContext, ?], and we have a better name for it - Compiler
2. init state of stateful effects can then be injected into program via ProgramContext when actually running Kleisli[IO, ProgramContext, ?]

• Id
• WriterT
• ReaderT/Kleisli
• StateT
• EitherT
• Http4sClient
• Doobie ConnectionIO
• fs2

It's very similar but just one more step to run the Kleisli

1. create Program dsl
2. compile Program into a Kleisli
3. run Kleisli in a context

e.g. our Program has lot of effects... WriterT, Http4sClient, ReaderT, IO, StateT and Doobie's ConnectionIO

few of them need to be stateful across all over the program like WriterT, StateT

 type Program[A] = Eff7[
      Http4sClient[IO, ?],
      WriterT[IO, Chain[String], ?],
      ReaderT[IO, Config, ?],
      IO,
      ConnectionIO,
      StateT[IO, Int, ?],
      Either[Throwable, ?],
      A
    ]
type ProgramF[A] = Free[Program, A]

EffX is predefined alias of type to construct multiple kind in EitherK

Now lets start using these effects to do our work

val program = for {
    config <- free[Program](Kleisli.ask[IO, Config])
    _ <- free[Program](
    GetStatus[IO](GET(Uri.uri("https://blog.oyanglul.us"))))
    _ <- free[Program](StateT.modify[IO, Int](1 + _))
    _ <- free[Program](StateT.modify[IO, Int](1 + _))
    state <- free[Program](StateT.get[IO, Int])
    _ <- free[Program](
    WriterT.tell[IO, Chain[String]](
      Chain.one("config: " + config.token)))
    resOrError <- free[Program](sql"""select true""".query[Boolean].unique)
    _ <- free[Program](
    resOrError.handleError(e => println(s"handle db error $e")))
    _ <- free[Program](IO(println(s"im IO...state: $state")))
} yield ()

if we compile our program, we should get a binary ProgramBin

import us.oyanglul.luci.compilers.io._
val binary = compile(program)

imagine that you have a binary of command line tool, when you run it you would probably need to provide some --args

same here, if you want to run ProgramBin, which is basically just a Kleisli, we need to provide args with is ProgramContext

run the program with real --args

val args = (httpclient ::
    logRef.tellInstance ::
    config ::
    Unit ::
    transactor ::
    stateRef.stateInstance ::
    Unit ::
    HNil)

binary.run(args)

for stateful WriterT and StateT here, we can get FunctorTell and MonadState instances from Ref[IO, ?] and inject them into the program via ProgramContext

each one corresponds to program's effect's context

1. binary for Http4sClient[IO, ?] needs Client[IO] to run
2. binary for WriterT[IO, Chain[String], ?] needs FuntorTell[IO, Chain[String]], presented by meow-mtl .tellInstance
3. binary for ReaderT[IO, Config, ?] needs Config to run
4. binary for IO needs nothing so Unit
5. binary for ConnectionIO needs Transactor[IO]
6. binary for StateT[IO, Int, ?] needs MonadState[IO, Int] to run, which presented here by meow-mtl from .stateInstance
7. binary for Either[Throwable, ?] needs nothing so Unit

creating a new compilable effect is pretty simple in 2 steps

This is nothing different from creating an effect data type for Free Monad

For instance, we need a s3 putObject Effect

import com.amazonaws.services.s3.model.PutObjectResult

sealed trait S3[A]

case class PutObject(bucketName: String, fileName: String, content: String)
    extends S3[PutObjectResult]

To create a compiler for new data type s3, we'll need to create an instance for type class Compiler

trait Compiler[F[_], E[_]] {
  type Env
  val compile: F ~> Kleisli[E, Env, ?]
}

We need a type of Env where the program needs to be compile. e.g. S3 need an AWS S3 Client

trait S3Compiler[E[_]] {
  implicit def s3Compiler(implicit F: Applicative[E]) = new Compiler[S3, E] {
    type Env = AmazonS3
    val compile = Lambda[S3 ~> Kleisli[E, Env, ?]] (_ match {
      case PutObject(bucketName, fileName, content) =>
        Kleisli(env => F.pure(env.putObject(bucketName, fileName, content)))
    })
  }
}

to be honest you don't need to make S3Compiler so generic since you may be the only person who using it. But it's a good practic to make every thing as genric as possible.

any way to use the generic effect, you can create a specific object just for IO(or Task of your choice)

object s3IoCompiler extends S3Compiler[IO]

and then import it to where you need to compile

import s3IoCompiler._

Or, simply extends it on the object or class you intent to compile your program

object Main extends S3Compiler[IO] with All{
  ...
  val binary = compile(program)
  ...
}