json-lenses is a library to query and update JSON data.
It has the following features
- Create queries similar to xpath in a native Scala DSL
- Retrieve selected elements
- Easily update selected elements of the immutable spray-json representation
- Selected elements can be of several cardinalities: scalar values, optional values, and sequences of values
- Experimental support for json-path syntax
If you use SBT you can include json-lenses in your project with
"net.virtual-void" %% "json-lenses" % "0.6.2"
(json-lenses supports spray-json 1.3.x. For support of an older spray-json version use json-lenses 0.5.4. See https://github.com/jrudolph/json-lenses/tree/v0.5.4-scala-2.11 for the old documentation).
Given this example json document:
val json = """
{ "store": {
"book": [
{ "category": "reference",
"author": "Nigel Rees",
"title": "Sayings of the Century",
"price": 8.95
},
{ "category": "fiction",
"author": "Evelyn Waugh",
"title": "Sword of Honour",
"price": 12.99,
"isbn": "0-553-21311-3"
}
],
"bicycle": {
"color": "red",
"price": 19.95
}
}
}""".parseJson
All authors in this document are addressed by
import spray.json.lenses.JsonLenses._
import spray.json.DefaultJsonProtocol._
val allAuthors = 'store / 'book / * / 'author
This is called a lens. You can use a lens to retrieve or update the addressed values.
val authorNames = json.extract[String](allAuthors)
To update values use one of the defined operations. To overwrite a value use set
, to change a value
based on the previous value use modify
.
// overwrite all authors' names to "John Doe"
val newJson1 = json.update(allAuthors ! set[String]("John Doe"))
// prepend authors' names with "Ms or Mr "
val newJson2 = json.update(allAuthors ! modify[String]("Ms or Mr " + _))
Here are other interesting queries on the example data:
// The author of the first book in the store
val firstAuthor = "store" / "book" / element(0) / "author"
// The titles of books more expensive than $ 10
val expensiveBookTitles = "store" / "book" / filter("price".is[Double](_ >= 10)) / "title"
// ISBN of all books that have one
val allIsbn = 'store / 'book / * / 'isbn.? // not all books have ISBNs, so the selection must be optional
The concept of lenses is a powerful concept not only applicable to json objects. A lens is an updatable, composable view into a data structure. There are simple lenses which provide just the functionality to "go one level deeper" in the data structure (for json: accessing fields of a json object or elements of a json array) and there are lenses which compose other lenses to allow a deeper view into the data structure.
See this answer on stack overflow and this presentation for more info on the general topic of lenses.
The json lenses in this project are specialized lenses to extract and update json data. A lens has this form (almost, in reality it's a bit more complicated):
// not the real code
trait Lens {
def retr: JsValue => JsValue
def updated(f: JsValue => JsValue)(parent: JsValue): JsValue
}
It has a method retr
which when called with a json value will return the value this lens points to.
The other method updated
takes a parent json value and returns an updated copy of it with function f
applied to the child value this lens points to.
In contrast to a more general lens input and output types of the json lens are fixed: both have to be an
instance of JsValue
.
The simple scheme introduced in the last section is actually too simple to support more than the absolute simplest lenses. One basic requirement is to extract a list of values or an optional value. Therefore, lenses are parameterized by the container type for the cardinality of the lens.
// still not the real code
trait Lens[M[_]] {
def retr: JsValue => M[JsValue]
// ...
Scalar lenses are of type Lens[Id]
(Id
being defined as the identity type constructor, so Id[T] == T
).
Optional lenses are of type Lens[Option]
. Lenses returning a sequence of values are of type Lens[Seq]
.
It's interesting what happens when two lenses of different cardinality are joined: The rule is to always return a lens
of the more general container type, i.e. the one with the greater cardinality. Joining two lenses of sequences of values
results in a flattened sequence of all the results. See the source code of JsonLenses.combine
and the Join
type class
to see how this is done.
To support proper error handling and recovery from errors (in some cases) failure is always assumed as a
possible outcome. This is reflected by returning an Either
value from almost all functions which may fail. The
real declaration of the Lens
type therefore looks more like this:
type Validated[T] = Either[Exception, T]
type SafeJsValue = Validated[JsValue]
trait Lens[M[_]] {
def retr: JsValue => Validated[M[JsValue]]
def updated(f: SafeJsValue => SafeJsValue)(parent: JsValue): SafeJsValue
}
The result of retr
is not always a Right(jsValue)
but may fail with Left(exception)
as well. The same
for updated
: the update operation may also fail. However, in the update case there are two other possibilities
encoded as well: Retrieval of a value may fail but the update operation f
may still succeed (for example, when
setting a previously undefined value) or the update operation itself fails in which case the complete update
operation fails as well.
When working with lenses you normally don't have to worry about the details but you can just choose and combine lenses from the following list.
-
field(name: String)
: This lens assumes that the target value is a json object and selects the field with the specified name. Because this is the most common lens there are shortcut implicit conversions defined fromString
andSymbol
values. -
optionalField(name: String)
: This lens assumes a JsObject and tries to select the field of the given name. If the field is missing it just isn't selected.field.?
is a shortcut for this lens (wherefield
should be an expression of type Symbol or String).
elements
or*
: This lens selects all the elements of a json array. If you combine this lens with another one you have to make sure that the next lens will match for all the elements of the array, otherwise it is an error. UseallMatching
if you want to exclude elements not matching nested lenses.element(idx: Int)
: Selects the element of a json array with the given index.find(predicate: JsValue => Boolean)
: Selects all elements of a json array which match a certain predicated. In the common case you don't want to work directly onJsValue
s so you can use lenses as well to define the predicate. UseLens.is[T](pred: T => Boolean)
to lift a predicate from the value level to theJsValue
level. E.g. use'fullName.is[String](_ startsWith "Joe")
to create a predicate which checks if a value is a json object and its fieldfullName
is a string starting with"Joe"
.allMatching(next: Lens)
: A combination ofcombine
andelements
. Selects elements matching thenext
lens and then applies thenext
lens.arrayOrSingletonAsArray
: Makes sure that the result is always a json array by interpreting a value that is no array as an singleton array containing just that value.
combine
: This lens combines two lenses by executing the second one on the result of the first one. Use it to access nested elements. Because its use is so common there's a shortcut operator,Lens./
, which you can use to 'chain' lenses: e.g. use'abc / 'def
access field 'def' inside the object in field 'abc' of the root json value passed to the lens.
Currently, there are these update operations defined:
set[T: JsonWriter](t: => T)
uses theJsonWriter
to set the value to a constant value.modify[T: JsonFormat](f: T => T)
applies an update function to an existing value. The value has to be serializable from and toT
.modifyOrDeleteField[T: JsonFormat](f: T => Option[T])
can be used in conjunction with theoptionalField
lens. It applies an update function to an existing value. If the function returnsSome(newValue)
the field value will be updated to the new value. If the function returnsNone
the field will be deleted.setOrUpdateField[T: JsonFormat](default: => T)(f: T => T)
can be used in conjunction with theoptionalField
lens. It allows to decide which value to set an maybe previously missing value to based on the previous state of the field.
To extract a value from json data using a lens you can use either lens.get[T](json)
or the equivalent
json.extract[T](lens)
which both throw an exception in case something fails or use lens.tryGet[T](json)
which returns an Either
value you can match on (or use Either
's functions) to check for errors.
To update a value use either (lens ! operation).apply(json)
or json.update(lens ! operation)
. You can
also use the fancy val newJson = json(lens) = "abc"
to set a value if you like that syntax. These operations
throw an exception in the error case. Use lens.updated(operation)(json)
to get an Either
result to process errors.
json-path support
Use JsonLenses.fromPath
which tries to create a lens from a json-path expression. Currently not all
of the syntax of json-path is supported. Please file an issue if something is missing.
You can find the documentation for the json-lenses API here.
- Richer set of operations (e.g. append and delete elements from arrays)
- ...
Please file an issue at the issue tracker if you need something.
Please use the spray-user mailing list to discuss json-lenses issues.
spray-json is licensed under APL 2.0.