#+DATE: <2016-03-20 Sun>
#+AUTHOR: Edd Steel
#+EMAIL: edd@eddsteel.com
#+title: Applying least power in Scala
#+OPTIONS: num:nil toc:nil author:nil date:nil
#+OPTIONS: ':t *:t -:t ::t <:t
#+OPTIONS: html-link-use-abs-url:nil html-postamble:auto html-head-include-default-style:nil
#+OPTIONS: html-preamble:t html-scripts:t html-style:t
#+OPTIONS: html5-fancy:t tex:t

#+HTML_HEAD: <link rel="stylesheet" href="../style.css">
#+HTML_HEAD: <link href="https://fonts.googleapis.com/css?family=Bitter&subset=latin,latin-ext" rel="stylesheet" type="text/css">
#+HTML_HEAD: <link rel="stylesheet" media="(max-width: 600px)" type="text/css" href="../style-under-600.css">
#+HTML_HEAD: <link rel="stylesheet" media="(max-width: 400px)" type="text/css" href="../style-under-400.css">
#+HTML_DOCTYPE: html5
#+HTML_CONTAINER: div
#+HTML_LINK_HOME: /
#+HTML_LINK_UP: ../posts
#+CREATOR: <a href="http://www.gnu.org/software/emacs/">Emacs</a> 24.5.2 (<a href="http://orgmode.org">Org</a> mode 8.3.4)
#+STARTUP: showeverything
#+LINK: tw   https://twitter.com/
#+LANGUAGE: en
#+EXCLUDE_TAGS: noexport
#
#
#
# To rehydrate, run org-babel-tangle, then org-export-dispatch, and choose an HTML option.
#

#+name: init
#+begin_src sh :exports results
mkdir -p lp >/dev/null 2>&1
#+end_src

#+RESULTS: init

I've learned a lot from the people I work with. One such person is
[[https://github.com/ujihisa][Uji]], who is always happy to give
functional programming advice, but more importantly, always happy to
explain the basis for it. There's usually a general and well-thought
out principle at work.

One rule comes up in discussion and code review so much it's now known
as "the Uji Principle". You may know it by a more descriptive
name. It's best explained by example, in this case incrementing the
values in a list.


#+name: ujiprinciple.scala
#+begin_src scala :exports both :results output table raw
  // The map way, apply a function to every element.
  //
  def incMap(l: List[Int]): List[Int] =
    l.map(_ + 1)

  // The fold way, accumulate a result by recursively combining
  // elements.
  //
  def incFold(l: List[Int]): List[Int] =
    l.foldRight(List.empty[Int]){ (next, acc) =>
      (next + 1) :: acc
    }

  // The recursive way, use the structure of the list to repeat an
  // operation on elements.
  //
  def incRecurse(l: List[Int]): List[Int] =
    l match {
      case Nil => Nil
      case (h::tl) => h + 1 :: incRecurse(tl)
    }

  // The mutation way, create a new list and populate it by looping
  // over the old one.
  //
  def incMutate(l: List[Int]): List[Int] = {
    var newList = List[Int]()
    for (i <- l) newList = newList :+ (i + 1)
    newList
  }

  // The shared mutable memory way (let's just plumb the depths here)
  //
  import scala.collection.mutable.ListBuffer
  object Store {
    var myInts: ListBuffer[Int] = _
  }
  def incSideEffect(l: List[Int]): List[Int] = {
    Store.myInts = ListBuffer.empty[Int]
    for (i <- l) Store.myInts += (i + 1)
    Store.myInts.toList
  }


  // Test they work, make a table of results.
  //
  println("|Name|Result|")
  println("|-+-|")
  println("|<l>|<c>|")
  test("map", incMap)
  test("fold right", incFold)
  test("recurse", incRecurse)
  test("mutate", incMutate)
  test("side effect", incSideEffect)

  //  Output a tick for yes, cross for no.
  //
  def test(name: String, f: List[Int] => List[Int]) = {
    val works = f(List(1,2,3)) == List(2,3,4)
    val result = if (works) "✔" else "✘"

    println(s"|$name| $result |")
  }
#+END_SRC

They all seem to work. Here's the output.

#+RESULTS: ujiprinciple.scala
| Name        | Result |
|-------------+--------|
| <l>         | <c>    |
| map         | ✔      |
| fold right  | ✔      |
| recurse     | ✔      |
| mutate      | ✔      |
| side effect | ✔      |

I think anyone who understands that code will say that the first
solution is better than the last.  They'd probably also agree the
solutions start with the best and progressively get worse.

So why is ~map~ a better solution than shared memory in this case? Or
even than ~foldRight~? The answer -- the principle -- is simple: it is
the most specific operation to solve the problem. You can rewrite any
use of ~map~ with ~foldRight~ but not the other way around, and this
relationship exists with every possible pair of operations; they are
ordered.

With this ordering, ~map~ is the most constrained. It's an abstraction
at just the right level. It's the least powerful thing that can
work. To apply the Uji principle to find the better of two options
simply start with the option that is the least powerful, and if it
works, you're done.


This is known in Computer Science and blogs as the Rule or Principle
of Least Power[fn:rules]. Here it is stated by the [[https://www.w3.org][W3C]] in 2001,
applied to choosing an implementation language.

#+begin_quote
When designing computer systems, one is often faced with a choice
between using a more or less powerful language for publishing
information, for expressing constraints, or for solving some
problem. This finding explores tradeoffs relating the choice of
language to reusability of information. The "Rule of Least Power"
suggests choosing the least powerful language suitable for a given
purpose.

                                       -- W3C, [[https://www.w3.org/2001/tag/doc/leastPower.html][Rule of Least Power]]
#+end_quote

* Rúnar Bjarnason's "Constraints liberate, liberties constrain"
  :PROPERTIES:
  :CUSTOM_ID: constraints-liberate
  :END:

Back to Scala, which has literally never been described as the least
powerful language suitable for a purpose. We know the principle
applies (ask Uji) but what do we mean by "power" here? A good
description was given by Rúnar Bjarnason in his [[https://www.youtube.com/watch?v=GqmsQeSzMdw][talk]] "Constraints
liberate, liberties constrain". You should watch it, it's great!

Rúnar talks about least privilege. Here, when we talk
about more powerful, we mean unconstrained, at the cost of
compositionality and modularity. "The more a system /can/ do, the less
we can predict what it /will/ do".

My ~map~ example is the least concrete -- it deals with an abstraction
at the right level to solve the problem, where implementation details
are provided by something else. To use Rúnar's language, it
"detonates" later than the shared memory, entirely concrete
implementation. Think about the effort to make the first example run
in parallel, compared to the last.

It's worth taking the time to get this straight in your mind;
counter-intuitively, a more abstract, more generic function is less
powerful according to this definition. That doesn't mean it's less
useful or less reusable, quite the opposite in fact, that's the point.

Of course you could go further, we would be relying on less "powerful",
more abstract mechanisms if our method signature was written
specifically in terms of "things that can be mapped over" (functors)
and "things that can take part in a binary associative operation"
(semigroups) and according to this principle that would be an
improvement.

* Li Haoyi's "Principle of Least Power"
  :PROPERTIES:
  :CUSTOM_ID: principle-of-least-power
  :END:

Recently Li Haoyi wrote [[http://www.lihaoyi.com/post/StrategicScalaStylePrincipleofLeastPower.html][a wonderful article]] describing what it means
to apply this principle to Scala development in general. Like the
element incrementer functions above, he orders various approaches by
power in categories:

- immutability
- published API
- data types
- error handling
- asynchronous return types
- dependency injection

** A Quibble
   :PROPERTIES:
   :CUSTOM_ID: plp-quibble
   :END:

It's a really good reference and I agree with almost everything in
it. However I would like to quibble with one point, and this is the
Internet so I will. [[http://www.lihaoyi.com/post/StrategicScalaStylePrincipleofLeastPower.html#comment-2513931369][Ken Scambler]] pointed it out too. "You should
always use built-in primitives and collections" might lead you to miss
out on some of the benefits of a type checker. If your function
expects a user ID, it should not accept a ~String~. Not unless the
complete works of Shakespeare is a valid user ID[fn:scambler]. And
this is completely in line with our version of what least power means
for Scala: choose the most constrained abstraction that will work.

** The best bit
   :PROPERTIES:
   :CUSTOM_ID: plp-best-bit
   :END:

But one thing stuck out to me as very definitely correct, and probably
controversial to many readers:

#+begin_quote
In general, if you find yourself passing an object to a method that
only calls a single method, consider making that method take a
FunctionN and passing in a lambda.

                    -- Li Haoyi, [[http://www.lihaoyi.com/post/StrategicScalaStylePrincipleofLeastPower.html#functions][Principle of Least Power]]
#+end_quote

This ties in very well with the [[http://www.lihaoyi.com/post/StrategicScalaStylePrincipleofLeastPower.html#dependency-injection][Dependency Injection]] section. Instead
of injecting dependencies through mix-ins and constructors, pass
things (i.e. probably functions) to the methods that need them. That
applies equally to test code, where suddenly mocks aren't nearly so
important.

I'd like to explore these ideas further because they can be directly
applied to the sort of code we write every day.  So then, what does he
mean? He means if your business logic requires access to a user's ID,
pass in a user ID, not a user. If your business logic needs to
constuct a ~Widget~ then use that to calculate some stuff, pass in a
~ID => Widget~ function that handles that construction, don't put the
logic in a trait that's self-typed to ~WidgetProvider~ instantiated in
a boot class along with widget constuction and seven unrelated things
~with WidgetSauce with WidgetRules with God with OhGod with
OhGodOhGodOhGod~.

But what of our ~AbstractServiceDecoratorComponent~? How can we show
off our enterprise test mocking libraries? What will become of our
inscrutable and many-layered cake?? I understand, it sounds
controversial. It will be OK.

I've been taking this idea to heart the past couple of weeks and I
have to say I think it's wonderful advice. It has made code and tests
clearer, quicker to write and easier to reason about. Alright, enough
exposition, this is Scala. Let's look at contrived code.

* A service example
  :PROPERTIES:
  :CUSTOM_ID: contrived-code
  :END:

#+name: webapp.scala
#+begin_src scala :exports none :tangle lp/webapp.scala :results silent
  package com.eddsteel.posts.leastpower

  import scala.util.{Try, Success}
  import scala.concurrent.Future

  package object webapp {
    type Request = String
    type Response = String
    type Action = (Request) => Future[Response]
    sealed trait Method
    case object POST extends Method

    trait Controller {
      def route(method: Method, path: String): Action
    }

    case class NotFoundException[A](a: A) extends RuntimeException(a.toString)
  }

  object WebApp {
    import webapp._

    def mkResponse[T](t: T): Try[Response] =
      Success(t.toString)

    def mkErrorResponse: PartialFunction[Throwable, Response] = {
      case _ => "An error occurred. Sad face."
    }


    def extractAndValidateUserId(req: Request): Try[Long] =
      Success(1L)

    def postFakeRequest(controller: Controller, method: Method, path: String, req: Request): Future[Response] =
      controller.route(method, path)(req)
  }
#+end_src

#+name: datetimeutils.scala
#+begin_src scala :exports none :tangle lp/datetimeutils.scala :results silent
  package com.eddsteel.posts.leastpower

  import java.time.{LocalDate, ZonedDateTime}

  object DateTimeUtils {
    private var overriddenNow: Option[ZonedDateTime] = None

    def now: ZonedDateTime = synchronized {
      overriddenNow.fold(ZonedDateTime.now)(identity)
    }

    def setDateTime(d: ZonedDateTime): Unit = synchronized {
      overriddenNow = Some(d)
    }

    def sameDay(one: LocalDate, two: LocalDate) =
      one.getYear == two.getYear &&
      one.getMonth == two.getMonth &&
      one.getDayOfMonth == two.getDayOfMonth
  }
#+end_src

#+name: services.scala
#+begin_src scala :exports none :tangle lp/services.scala :results silent
  package com.eddsteel.posts.leastpower
  package services

  import model._

  import scala.concurrent.Future

  trait UserServiceComponent {
    def userService: UserService
  }

  trait UserService {
    def getById(id: ID): Future[Option[User]]
  }

  trait GiftServiceComponent {
    def giftService: GiftService
  }

  trait GiftService {
    def order(userId: ID, favoriteThing: String): Future[GiftService.Response]
  }

  object GiftService {
    type Response = (String, Option[ID])
  }

#+end_src

Our imaginary startup has been running in stealth mode for several
months, and our rockstar ninja pirate back-end developers have already
built a fully reactive user service. The system is up and gathering
user information. Here's our user model, with the business-critical
information we store:

#+name: model.scala
#+begin_src scala :tangle lp/model.scala :results silent
  package com.eddsteel.posts.leastpower

  import java.time.{LocalDate => Date, ZoneOffset => TimeZone}

  package object model {

    final case class User(
      id: ID, birthday: Date, timezone: TimeZone,
      favoriteThing: String, pets: Set[Pet], whichFriendAreYou: Friend)

    type ID = Long
    type Pet = String

    sealed trait Friend
    object Friends {
      case object Joey extends Friend
      case object Phoebe extends Friend
      case object Chandler extends Friend
      case object Rachel extends Friend
      case object Monica extends Friend
      // we definitely don't need Ross in our MVP. No one likes you Ross.
    }
  }
#+end_src

Users are available to the rest of the system over a ~UserService~
client. We use that to call the service. Given an ~ID~ it will return
us a ~Future[Option[Long]]~, that is an asynchronous value that may be
nothing (if the user is not found) and may be a user. In addition,
~Future~ values model potential failure. Instead of an ~Option[User]~
we may find our future has failed, in which case it holds the cause
~Throwable~. ~Future~ is useful for this example but if it offends you
please feel free to substitute your concurrency primitive of choice.

** Let's build an endpoint
   :PROPERTIES:
   :CUSTOM_ID: version-1
   :END:


Good news! We have pivoted, again, and will now be offering a gift
delivery service to registered users of our app. This will
generate revenue, I mean obviously.

In order to do this, we have built a gift service and a ~GiftService~
client[fn:giftservice]. Now we just need to tie that together in a web app our
front-end will call. Don't worry, they'll only call it once per day
per user, and delivery is instant!

The front-end will =POST= to =/users/:id/gift= and expect us to
order a gift of the user's favourite thing, if it's the user's
birthday (via the gift service). It will expect to receive a status
message (for the web user to read) and the gift's ID (for the
machines) if a gift was ordered. Simple.

#+name: service-endpoints-v1.scala
#+begin_src scala :tangle lp/service-endpoints-v1.scala
  package com.eddsteel.posts.leastpower
  package endpoints

  import model._
  import services._
  import webapp.{Request, Response, NotFoundException}

  import scala.concurrent.{ExecutionContext, Future}
  import scala.util.Try
  import java.time.LocalDate

  /** Endpoints for our Web app.
   *
   *  The important feature: given a user ID, will order that user a gift
   *  if it's their birthday.
   *
   */
  trait EndpointsV1 {
    this: UserServiceComponent with GiftServiceComponent =>
    import EndpointsV1._

    def postGift(req: Request)(implicit ec: ExecutionContext): Future[Response] = {
      val response = for {
        // extract ID from request path
        userId <- Future.fromTry(WebApp.extractAndValidateUserId(req))
        // retrieve the user
        maybeUser <- userService.getById(userId)
        // bail with 404 if we have no user.
        user = maybeUser.getOrElse(throw NotFoundException(userId))
        // check if it's the user's birthday
        isBirthday = isToday(user.birthday)
        // call gift service if necessary
        gift <- if (isBirthday) giftService.order(userId, user.favoriteThing)
                else Future.successful(("not today", None))
        // build success response
        response <- Future.fromTry(WebApp.mkResponse(gift))
      } yield response

      // build error response
      response.recover(WebApp.mkErrorResponse)
    }
  }

  object EndpointsV1 {
    private def isToday(birthday: LocalDate): Boolean = {
      // using this utility helps us write tests.
      val today = DateTimeUtils.now.toLocalDate

      today.getDayOfMonth == birthday.getDayOfMonth &&
      today.getMonth == birthday.getMonth
    }

  }
#+end_src

For the purpose of discussion let's assume, let's just say, that the
framework "WebApp" is separate from this service and well
tested.[fn:framework] Hopefully with a little imagination this code is
fairly self-explanatory a flow of information from request, through
services to response. We get the ID, we get the user, if the user's
birthday is today we get the gift sent, and then we respond. See below
for more information.[fn:code]

The ~isToday~, ~mkResponse~, ~mkErrorResponse~,
~extractAndValidateUserId~ methods are used and tested elsewhere,
ditto the gruesome details of routing and data validation. In this
context, the only new thing is the middle few lines of ~postGift~.

Still, we're professionals, and we always practice responsible testing
for our systems. We certainly want to test those lines. Three, four
lines won't need much test code.

#+name: test_service-endpoints-v1.scala
#+begin_src scala :tangle lp/test_service-endpoints-v1.scala
  package com.eddsteel.posts.leastpower
  package endpoints

  import model._
  import services._
  import webapp._

  import org.scalatest.{FlatSpec, Matchers}
  import org.scalatest.concurrent.ScalaFutures
  import org.scalatest.mock.MockitoSugar
  import org.mockito.Mockito.{when, verify, RETURNS_SMART_NULLS}

  import scala.concurrent.{Future, ExecutionContext}
  import java.time.{ZoneOffset, LocalDate, ZonedDateTime, ZoneId}

  class EndpointTestsV1 extends FlatSpec with Matchers with MockitoSugar with ScalaFutures {

    import ExecutionContext.Implicits.global

    // mock out our dependencies.
    // if we don't provide "RETURNS_SMART_NULLS" no effort will be taken
    // to explain the NullPointerExceptions thrown by misconfigured mocks.
    //
    val mockUserService = mock[UserService](RETURNS_SMART_NULLS)
    val mockGiftService = mock[GiftService](RETURNS_SMART_NULLS)
    val controller = new Controller with EndpointsV1 with UserServiceComponent with GiftServiceComponent {
      override val userService = mockUserService
      override val giftService = mockGiftService

      // all routes go to the endpoint we're testing.
      override def route(method: Method, path: String): Action = postGift
    }

    "POST /users/:id/gift" should "order a gift for a user on their birthday" in {
      // make the world ready for our test
      //
      DateTimeUtils.setDateTime(ZonedDateTime.of(2015, 1, 1, 0, 0, 0, 0, ZoneOffset.UTC))

      // mock the return of a user that contains the right
      // information (and a bunch of irrelevant stuff too, because we
      // have to)
      //
      when(mockUserService.getById(1L)).thenReturn(
        Future.successful(Some(
          User(id = 1, birthday = LocalDate.of(1980, 1, 1), favoriteThing = "booze",
               timezone = ZoneOffset.UTC, pets = Set.empty, whichFriendAreYou = Friends.Phoebe))))

      // mock the gift service to just return a message saying it
      // successfully ordered the favorite thing (we're not testing this
      // service)
      //
      when(mockGiftService.order(1L, "booze")).thenReturn(
        Future.successful(
          ("booze ordered for user 1", Some(1L))))

      // Send the request through the routing logic to the endpoint to test the endpoint
      // logic.
      //
      val response = WebApp.postFakeRequest(controller, POST, "/users/1/gift", "")

      response.futureValue.shouldEqual("(booze ordered for user 1,Some(1))")
      verify(mockUserService).getById(1L)
      verify(mockGiftService).order(1L, "booze") // check the gift actually was ordered through the service
    }
  }
#+end_src

#+name: test-v1
#+begin_src sh :exports results :results output :dir lp
sbt "testOnly *V1" | tail -n 9
#+end_src

#+RESULTS: test-v1
: [info] EndpointTestsV1:
: [info] POST /users/:id/gift
: [info] - should order a gift for a user on their birthday
: [info] Run completed in 525 milliseconds.
: [info] Total number of tests run: 1
: [info] Suites: completed 1, aborted 0
: [info] Tests: succeeded 1, failed 0, canceled 0, ignored 0, pending 0
: [info] All tests passed.
: [success] Total time: 1 s, completed Mar 19, 2016 11:22:58 AM

#+RESULTS:

Well.

Given a text-processing problem a developer might choose to use
regular expressions, and now she has two problems. Given a testing
problem a developer might choose to use mocks, and now blood is
gushing from her eyesockets[fn:eyesockets]. Let's take a moment and
look at why they were necessary.

The logic we're testing needs to get a user based on an ID, so we
mock the service that does that. Also we need to stub out the rest of
the model, even though we don't want or need those details. And based
on the logic we are testing a gift should be ordered, so we mock that
service too. In order to test the logic, because it is buried inside
an endpoint action, we need to construct a request, and pass it
through our framework in order to get something we can inspect.

In order to test time-specific logic, as in "is it the user's birthday
today?" we have a utility to fetch "now". Usually it returns the
current time, but in tests we can override it.

Maybe we can lose some of this.


** Narrow the scope of test
  :PROPERTIES:
  :CUSTOM_ID: version-2
  :END:

We're constructing a whole fake set of endpoints in order to test
specific behaviour. Our test covers code for routing, error recovery,
and service calls but we're not actually testing it. Also our eye
sockets are bleeding quite a lot. Let's remember our framework is well
covered by existing tests, and extract just the logic we want to
test. While we're at it, let's leave out the user fetching. We're not
testing that. We'll want to test error handling of this logic, but not
in this test.

While we're at that, let's start thinking least-power and pass in the
dependencies like service clients instead of plucking them from the
scope of the local instance. Data in, data out would be far easier to
deal with[fn:pedantry].

Behold, ~maybeSendGift~!

#+begin_src scala :tangle lp/service-endpoints-v2.scala
  package com.eddsteel.posts.leastpower
  package endpoints

  import model._
  import services._
  import webapp.{Request, Response, NotFoundException}

  import scala.concurrent.{ExecutionContext, Future}
  import scala.util.Try
  import java.time.LocalDate

  /** Let's change the test to ignore the request/response/routing stuff and just test the business logic.
    *
    * It would be easier to test this logic if we made the dependencies
    * explicit (method params), and moved it out to the companion object.
    */
  trait EndpointsV2 {
    this: UserServiceComponent with GiftServiceComponent =>
    import EndpointsV2._

    def postGift(req: Request)(implicit ec: ExecutionContext): Future[Response] = {
      val response = for {
        // extract ID from request path
        userId <- Future.fromTry(WebApp.extractAndValidateUserId(req))
        // retrieve the user
        maybeUser <- userService.getById(userId)
        // bail with 404 if we have no user.
        user = maybeUser.getOrElse(throw NotFoundException(userId))
        // check if it's the user's birthday, and maybe send them a gift.
        gift <- maybeSendGift(user, giftService)
        // build success response
        response <- Future.fromTry(WebApp.mkResponse(gift))
      } yield response

      // build error response
      response.recover(WebApp.mkErrorResponse)
    }
  }

  object EndpointsV2 {
    private def isToday(birthday: LocalDate): Boolean = {
      // using this utility helps us write tests.
      val today = DateTimeUtils.now.toLocalDate

      today.getDayOfMonth == birthday.getDayOfMonth &&
      today.getMonth == birthday.getMonth
    }

    private[endpoints] def maybeSendGift(user: User, giftService: GiftService) = {
      val birthdayToday = isToday(user.birthday)
      if (birthdayToday) giftService.order(user.id, user.favoriteThing)
      else Future.successful(("not today", None))
    }
  }
#+end_src

Let's see how much test we can throw away.

#+begin_src scala :tangle lp/test_service-endpoints-v2.scala
    package com.eddsteel.posts.leastpower
    package endpoints

    import model._
    import services._
    import webapp._

    import org.scalatest.{FlatSpec, Matchers}
    import org.scalatest.concurrent.ScalaFutures
    import org.scalatest.mock.MockitoSugar
    import org.mockito.Mockito.{when, verify, RETURNS_SMART_NULLS}

    import scala.concurrent.{Future, ExecutionContext}
    import java.time.{ZoneOffset, LocalDate, ZonedDateTime, ZoneId}

    class EndpointTestsV2 extends FlatSpec with Matchers with MockitoSugar with ScalaFutures {

      import ExecutionContext.Implicits.global

      val mockGiftService = mock[GiftService](RETURNS_SMART_NULLS)

      "maybeSendGift" should "order a gift for a user on their birthday" in {
        DateTimeUtils.setDateTime(ZonedDateTime.of(2015, 1, 1, 0, 0, 0, 0, ZoneOffset.UTC))

        val fakeUser =
          User(id = 1, birthday = LocalDate.of(1980, 1, 1), favoriteThing = "booze",
               timezone = ZoneOffset.UTC, pets = Set.empty, whichFriendAreYou = Friends.Phoebe)

        when(mockGiftService.order(1L, "booze")).thenReturn(
          Future.successful(("booze ordered for user 1", Some(1L))))

        val response = EndpointsV2.maybeSendGift(fakeUser, mockGiftService)
        response.futureValue.shouldEqual(("booze ordered for user 1", Some(1L)))
        verify(mockGiftService).order(1L, "booze") // check the gift actually was ordered through the service
      }
    }
#+end_src

#+name: test-v2
#+begin_src sh :exports results :results output :dir lp
sbt "testOnly *V2" | tail -n 9
#+end_src

#+RESULTS: test-v2
: [info] EndpointTestsV2:
: [info] maybeSendGift
: [info] - should order a gift for a user on their birthday
: [info] Run completed in 410 milliseconds.
: [info] Total number of tests run: 1
: [info] Suites: completed 1, aborted 0
: [info] Tests: succeeded 1, failed 0, canceled 0, ignored 0, pending 0
: [info] All tests passed.
: [success] Total time: 1 s, completed Mar 19, 2016 11:23:26 AM

Wow, that test is like half the size, and still covers what we needed!

Jessica Kerr wrote a great guide on [[http://engineering.monsanto.com/2015/07/28/avoiding-mocks/][writing tests without mocks]], and
we've used the "flow of data" approach to remove the user service
mock. Unfortunately we can't avoid mocking the gift service by
pre-ordering a gift, because we don't know outside of ~maybeSendGift~
if we want one.

Still let's count our blessings:

1. Don't have to build a fake endpoints object or pretend we know how the framework works.
2. We don't have to fear any future change to endpoints, our method stands alone.
3. We don't have to mock a user service, we just ask for a user.
4. Our eyesocket bleed is down to a low trickle.

Four blessings! And we're not done yet. There's still a mock to
replace, and the article also describes a "port and adapters approach"
which we'll see ties in well with the least power principle.

The other distasteful part of this code is that because we are
exchanging domain objects, a whole bunch of unneccessary information
is being carried along with what we care about. Why must our test
specify the test user most resembles Phoebe? Our test doesn't care,
but the compiler does. The compiler does because the data model does,
because four versions ago it had business value. Talk about incidental
complexity.

Additionally, our convenient "now" utility relies on some fairly
powerful scary stuff. It stops the world and sets universal
time. There must be a better way.

** Reduce Power
  :PROPERTIES:
  :CUSTOM_ID: version-3
  :END:

Let's apply the principle some more. That means:

- replace compound values with single values we actually need.
- replace objects whose methods we use with functions
- replace "dynamic variables" with method parameters

The first is obvious, the second two less so, but we'll see they give us a lot.

#+begin_src scala :tangle lp/service-endpoints-v3.scala
    package com.eddsteel.posts.leastpower
    package endpoints

    import model._
    import services._
    import webapp.{Request, Response, NotFoundException}

    import scala.concurrent.{ExecutionContext, Future}
    import scala.util.Try
    import java.time.{LocalDateTime, LocalDate}

     /* maybeSendGift still has some dependencies (in the parameters or
      * environment)
      *
      * - gift service (in order to call the order method)
      * - date time utils.
      *
      * By the principle of least power, let's pass those into the method
      * as functions instead.  let's also use plain values and functions
      * rather than containers of plain values and methods.
      */
     trait EndpointsV3 {
       this: UserServiceComponent with GiftServiceComponent =>
       import EndpointsV3._

       def postGift(req: Request)(implicit ec: ExecutionContext): Future[Response] = {
         val response = for {
           // extract ID from request path
           userId <- Future.fromTry(WebApp.extractAndValidateUserId(req))
           // retrieve the user
           maybeUser <- userService.getById(userId)
           // bail with 404 if we have no user.
           user = maybeUser.getOrElse(throw NotFoundException(userId))
           // check if it's the user's birthday, and maybe send them a gift
           gift <- maybeSendGift(user.id, user.birthday, user.favoriteThing,
                                 LocalDateTime.now, giftService.order)
           // build success response
           response <- Future.fromTry(WebApp.mkResponse(gift))
         } yield response

         // build error response
         response.recover(WebApp.mkErrorResponse)
       }
     }

     object EndpointsV3 {
       def isToday(birthday: LocalDate, now: => LocalDateTime) =
         now.getDayOfMonth == birthday.getDayOfMonth &&
         now.getMonth == birthday.getMonth

       def maybeSendGift(userId: ID, birthday: LocalDate, favoriteThing: String,
                         now: => LocalDateTime,
                         orderGift: (Long, String) => Future[(String, Option[Long])])
                        (implicit ec: ExecutionContext) = {
         val birthdayToday = isToday(birthday, now)
         if (birthdayToday) orderGift(userId, favoriteThing)
         else Future.successful(("not today", None))
       }
     }
#+end_src

We've added some overhead to our maybeSendGift. We've chosen more
constraints. For example, instead of sending a ~GiftService~ the
endpoint now sends a specific gift posting function (which
happens to use a gift service). This is straight out of the Li
playbook, and also what Kerr described as the "ports and adapters
approach". It's is slightly more code, and the signature of
~maybeSendGift~ is now bigger than the body. On the other hand, we
know as aspiring functional programmers that's the better way around
anyway. Also, we've lost some code in ~isToday~ as we no longer need
to use the powerful, timezone-aware utility. And we haven't even got
to the test yet.

#+begin_src scala :tangle lp/test_service-endpoints-v3.scala
  package com.eddsteel.posts.leastpower
  package endpoints

  import model._
  import services._
  import webapp._

  import org.scalatest.{FlatSpec, Matchers}
  import org.scalatest.concurrent.ScalaFutures

  import scala.concurrent.{Future, ExecutionContext}
  import java.time.{LocalDateTime, LocalDate}

  class EndpointsV3Test extends FlatSpec with Matchers with ScalaFutures {
    import ExecutionContext.Implicits.global

    "maybeSendGift" should "order a gift for a user on their birthday" in {

      def successfulGiftOrder(id: Long, thing: String) =
        Future.successful((s"$thing ordered for user $id", Some(id)))

      val result =
        EndpointsV3.maybeSendGift(1, LocalDate.of(1980, 1, 1), "booze",
                                  LocalDateTime.of(2016, 1, 1, 0, 0, 0, 0),
                                  successfulGiftOrder)

      result.futureValue.shouldEqual(("booze ordered for user 1", Some(1)))

    }
  }
#+END_SRC

#+name: test-v3
#+begin_src sh :exports results :results output :dir lp
sbt "testOnly *V3Test" | tail -n 9
#+end_src

#+RESULTS: test-v3
: [info] EndpointsV3Test:
: [info] maybeSendGift
: [info] - should order a gift for a user on their birthday
: [info] Run completed in 291 milliseconds.
: [info] Total number of tests run: 1
: [info] Suites: completed 1, aborted 0
: [info] Tests: succeeded 1, failed 0, canceled 0, ignored 0, pending 0
: [info] All tests passed.
: [success] Total time: 1 s, completed Mar 19, 2016 11:23:41 AM


Boom! Quite a bit shorter. Our test is now 3 (long) lines and can live
entirely outside our service framework. We're dealing with simple data
and functions which opens up opportunites like generative testing that
would have been very painful before.

Even better, despite the dependency on some sort of gift ordering
logic, we don't need to mock anything. We don't need to mock anything!

But don't we need mocks, not fakes? what about validating our gift service was
called? There's no need, because the only way for it to produce the
expected output is for it to be called with the correct parameters.

There's still room for improvement in our code -- we're still using
mix-ins to bring dependent services into our endpoints code. If we can
change our service framework, we should look at that next. Is
"exercise for the reader" a cop-out? We'll leave that as an exercise
for the reader.

* Worth it?
  :PROPERTIES:
  :CUSTOM_ID: worth-it
  :END:

Hopefully you want code more like example 3 than example 1, if so I
think you'll agree the principle of least power is a useful guide to
get you there. Let's summarize.

- The Principle of Least Power can help thinking about the complexity
  of your code, and reducing it.
- Following it can make your code less tightly coupled, easier to
  test, and easier to reason about.
- The cost is extra function parameters (which can be mitigated as
  necessary with case classes) and an extra level of indirection.
- Smart use of names and types can make the code just as expressive
  and clear as before (more so, actually),
- Remember that when it comes to abstraction, constraints liberate and liberties constrain.

Really this could be summarised as "prefer plain data and pure
functions, and the pain will go away", but hopefully the noisy details
help convince you of that.

* Ack
  :PROPERTIES:
  :CUSTOM_ID: ack
  :END:

Thanks to [[tw:li_haoyi][Li Haoyi]], [[tw:runarorama][Rúnar Bjarnason]], [[tw:jessitron][Jessica Kerr]], [[tw:kenscambler][Ken Scambler]] and
[[tw:ujm][Uji]]. This is based on some ideas I talked about at a recent Scala
Guild meeting at Hootsuite, which covered the "Principle of Least
Power". At the same guild meeting my colleague [[tw:typesake][Johnny]] presented
covering the "Constraints liberate, liberties constrain" talk, and my
colleague [[tw:ryanheinen][Ryan]] reminded me I'd linked to "Testing without Mocking"
before. You could read more about [[http://code.hootsuite.com/][Hootsuite]] and [[http://code.hootsuite.com/guilds/][Guilds]]. Needless to
say I don't speak for my employer, or those I've quoted here. Any
errors or inaccuracies in this post are mine. Thanks also to Emacs and
org-mode. You routinely blow my mind.




#+name: buildsbt
#+begin_src txt :tangle lp/build.sbt :exports none
name := "least-power"

scalaVersion := "2.11.7"

val testSrcs = "test_*.scala"
scalaSource in Test := baseDirectory.value
includeFilter in Test := testSrcs
excludeFilter in Compile := testSrcs
excludeFilter in Test := ""

libraryDependencies ++= Seq(
  "org.scalatest" %% "scalatest" % "2.2.6",
  "org.mockito" % "mockito-all" % "1.8.4"
)
#+end_src


* Source
  :PROPERTIES:
  :CUSTOM_ID: source
  :END:

#+name: zip-code
#+begin_src sh :exports results :dir lp
tar zcf leastpower-code.tar.gz build.sbt *.scala
#+end_src

#+RESULTS: zip-code

#+name: htmlize
#+begin_src emacs-lisp :exports results :results output raw replace :dir lp
  (require 'htmlize)
  (progn
    (htmlize-file "../least-power.org")
    (princ "  - =least-power.org= ([[file:least-power.org.html][view]])([[file:least-power.org][download]])\n")
    (mapcar
     (lambda (file)
       (let* ((html  (concat file ".html"))
              (htmll (concat "file:lp/" html))
              (filel (concat "file:lp/" file)))
         (htmlize-file file html)
         (princ
          (concat "  - =" file "= ([[" htmll "][view]])" " ([[" filel "][download]])\n"))))
     (cons "build.sbt" (file-expand-wildcards "*.scala")))

     (princ "  - [[file:lp/leastpower-code.tar.gz][=leastpower-code.tar.gz=]]\n"))
#+end_src
#+RESULTS: htmlize
  - =least-power.org= ([[file:least-power.org.html][view]])([[file:least-power.org][download]])
  - =build.sbt= ([[file:lp/build.sbt.html][view]]) ([[file:lp/build.sbt][download]])
  - =datetimeutils.scala= ([[file:lp/datetimeutils.scala.html][view]]) ([[file:lp/datetimeutils.scala][download]])
  - =model.scala= ([[file:lp/model.scala.html][view]]) ([[file:lp/model.scala][download]])
  - =service-endpoints-v1.scala= ([[file:lp/service-endpoints-v1.scala.html][view]]) ([[file:lp/service-endpoints-v1.scala][download]])
  - =service-endpoints-v2.scala= ([[file:lp/service-endpoints-v2.scala.html][view]]) ([[file:lp/service-endpoints-v2.scala][download]])
  - =service-endpoints-v3.scala= ([[file:lp/service-endpoints-v3.scala.html][view]]) ([[file:lp/service-endpoints-v3.scala][download]])
  - =services.scala= ([[file:lp/services.scala.html][view]]) ([[file:lp/services.scala][download]])
  - =test_service-endpoints-v1.scala= ([[file:lp/test_service-endpoints-v1.scala.html][view]]) ([[file:lp/test_service-endpoints-v1.scala][download]])
  - =test_service-endpoints-v2.scala= ([[file:lp/test_service-endpoints-v2.scala.html][view]]) ([[file:lp/test_service-endpoints-v2.scala][download]])
  - =test_service-endpoints-v3.scala= ([[file:lp/test_service-endpoints-v3.scala.html][view]]) ([[file:lp/test_service-endpoints-v3.scala][download]])
  - =webapp.scala= ([[file:lp/webapp.scala.html][view]]) ([[file:lp/webapp.scala][download]])
  - [[file:lp/leastpower-code.tar.gz][=leastpower-code.tar.gz=]]

#+name: clean
#+begin_src sh :exports results :dir lp
sbt clean >/dev/null 2>&1
rm -r project/target >/dev/null 2>&1
#+end_src

#+RESULTS: clean


* [0/2] TASKS                                                      :noexport:
** TODO upload and post index
** TODO what's up with the link to this file being to html?

* Footnotes

[fn:rules] A similar rule in Information Security is the rule of least
privilege, which is useful for similar, but distinct, reasons).

[fn:scambler]  And now I'm directly plagiarizing him. Do read the comment.


[fn:giftservice] ~GiftService~ provides ~orderGift~, of type ~(ID,
String) => Future[GiftService.Response]~. Given a description of a
gift and the recipient's ID, it will order a gift for that person. The
response contains what the webapp client needs, we'll pass it through.

[fn:pedantry] I know, we're not actually there yet. Keep going.

[fn:framework]  For the sake of example and NIH, we'll imagine the
service was written with a custom
[[file:lp/webapp.scala.html][library]], which resembles a blend of
scalatra, Play, http4s, spray etc for its own unique choice of
abstractions. Yes this code compiles, no it doesn't serve HTTP
traffic.

[fn:code] Let's briefly cover what this code does. It creates a trait
with some service dependencies, containing a single HTTP action. The
action, using these dependencies, accepts a user ID, gets user
information, checks if the user's birthday is today, and if so uses
the gift service to order that user a gift. Functions that require no
access to dependencies or shared state live in the companion object,
so no instance-specific information creeps in.\\
\\
The endpoint's flow is modelled as a for-comprehension over
~scala.concurrent.Future~, flatmapping over future values means that
as results become available the functions that use them are run (maybe
on other threads). Each line relies on values retrieved in previous
ones, so though these calls may be asynchronous and non-blocking they
must happen sequentially. Those that are synchronous are simply
wrapped, i.e. no thread switching takes place, but the compiler is
assured it can treat these immediately available values as future
values. If any step produces a failed future, the `flatMap` calls will
not run the following lines (flatmap on a failed future is simply that
failed future) and the endpoint recovers such a failure into an
appropriate response.

[fn:eyesockets] I think the best argument against mocking frameworks
is the dreary experience of using them. I'm not hating on Mockito
specifically, here, but the whole miserable idea. If you don't agree I
invite you to try a mocking framework for yourself. See? Sorry about
your eyesockets.