Tooling that enables converting to and from Smithy.
Note: this library is published to work on Java 8 and above. However, you will need to use Java 11 or above to work on the library as a contributor. This is due to some of the build flags that we use.
The smithytranslate
CLI will recursively go through all child directories of the
input directory provided and format any Smithy files it finds. The output
> smithytranslate format --help
Usage: smithytranslate format [--no-clobber] <path to Smithy file or directory containing Smithy files>...
validates and formats smithy files
Options and flags:
--help
Display this help text.
--no-clobber
dont overwrite existing file instead create a new file with the word 'formatted' appended so test.smithy -> test_formatted.smithy
- The formatter is based off the ABNF defined at Smithy-Idl-ABNF
- The formatter assumes the file is a valid Smithy file and must be able to pass the Model Assembler validation , otherwise it will return an error
- use --no-clobber to create a new file to avoid overwriting the original file
- actual formatting rules are still WIP and will be updated as the formatter is developed
Throughout smithytranslate you will see references to alloy. Alloy is a lightweight library that houses some common smithy shapes that are used across our open source projects such as smithy4s. This is to provide better interoperability between our tools at a lower cost to end users.
You will need to install coursier, an artifact fetching library, in order to install the CLI.
coursier install --channel https://disneystreaming.github.io/coursier.json smithytranslate
Run smithytranslate --help
for usage information.
The smithytranslate
CLI will recursively go through all child directories of the
input directory provided and convert any openapi files ending with an extension of yaml
,
yml
, or json
.
> smithytranslate openapi-to-smithy --help
Usage: smithytranslate openapi-to-smithy --input <path> [--input <path>]... [--verboseNames] [--failOnValidationErrors] [--useEnumTraitSyntax] [--outputJson] <directory>
Take Open API specs as input and produce Smithy files as output.
Options and flags:
--help
Display this help text.
--input <path>, -i <path>
input source files
--verbose-names
If set, names of shapes not be simplified and will be as verbose as possible
--validate-input
If set, abort the conversion if any input specs contains a validation error
--validate-output
If set, abort the conversion if any produced smithy spec contains a validation error
--enum-trait-syntax
output enum types with the smithy v1 enum trait (deprecated) syntax
--json-output
changes output format to be json representations of the smithy models
Run smithytranslate openapi-to-smithy --help
for more usage information.
Because Smithy is a more constrained format than OpenAPI, this conversion is partial. This means that a best effort is made to translate all possible aspects of OpenAPI into Smithy and errors are outputted when something cannot be translated. When errors are encountered, the conversion still makes a best effort at converting everything else. This way, as much of the specification will be translated automatically and the user can decide how to translate the rest.
OpenAPI 2.x and 3.x are supported as input formats to this converter.
Below are examples of how Smithy Translate converts various OpenAPI constructs into Smithy.
OpenAPI Base Type | OpenAPI Format | Smithy Shape | Smithy Trait(s) |
---|---|---|---|
string | String | ||
string | timestamp | Timestamp | |
string | date-time | Timestamp | @timestampFormat("date-time") |
string | date | String | alloy#dateFormat |
string | uuid | alloy#UUID | |
string | binary | Blob | |
string | byte | Blob | |
string | password | String | @sensitive |
number | float | Float | |
number | double | Double | |
number | double | Double | |
number | Double | ||
integer | int16 | Short | |
integer | Integer | ||
integer | int32 | Integer | |
integer | int64 | Long | |
boolean | Boolean | ||
object | (empty properties) | Document |
OpenAPI:
openapi: '3.0.'
info:
title: doc
version: 1.0.0
paths: {}
components:
schemas:
Testing:
type: object
properties:
myString:
type: string
my_int:
type: integer
required:
- myString
Smithy:
structure Testing {
@required
myString: String
my_int: Integer
}
Required properties and nested structures are both supported.
Any properties in the input structure that begin with a number will be prefixed by the letter n
. This is because smithy does not allow for member names to begin with a number. You can change this with post-processing if you want a different change to be made to names of this nature. Note that this extra n
will not impact JSON encoding/decoding because we also attach the JsonName Smithy trait to these properties. The same thing happens if the member name contains a hyphen. In this case, hyphens are replaced with underscores and a jsonName
trait is once again added. Note that if the field is a header or query parameter, the jsonName
annotation is not added since httpHeader
or httpQuery
is used instead.
OpenAPI:
openapi: '3.0.'
info:
title: doc
version: 1.0.0
paths: {}
components:
schemas:
Testing:
type: object
properties:
12_twelve:
type: string
X-something:
type: string
Smithy:
structure Testing {
@jsonName("12_twelve")
n12_twelve: String
@jsonName("X-something")
X_something: String
}
Smithy Translate will convert allOfs from OpenAPI into structures with mixins in smithy where possible. AllOfs in OpenAPI have references to other types which compose the current type. We refer to these as "parents" or "parent types" below. There are three possibilities when converting allOfs to smithy shapes:
- The parent structures are only ever used as mixins
OpenAPI:
openapi: '3.0.'
info:
title: doc
version: 1.0.0
paths: {}
components:
schemas:
One:
type: object
properties:
one:
type: string
Two:
type: object
properties:
two:
type: string
Three:
type: object
allOf:
- $ref: "#/components/schemas/One"
- $ref: "#/components/schemas/Two"
Smithy:
@mixin
structure One {
one: String
}
@mixin
structure Two {
two: String
}
structure Three with [One, Two] {}
Here we can see that both parents, One
and Two
are converted into mixins and used as such on Three
.
- The parents structures are used as mixins and referenced as member targets
OpenAPI:
openapi: '3.0.'
info:
title: doc
version: 1.0.0
paths: {}
components:
schemas:
One:
type: object
properties:
one:
type: string
Two:
type: object
allOf:
- $ref: "#/components/schemas/One"
Three:
type: object
properties:
one:
$ref: "#/components/schemas/One"
Smithy:
@mixin
structure OneMixin {
one: String
}
structure One with [OneMixin] {}
structure Two with [OneMixin] {}
structure Three {
one: One
}
Here One
is used as a target of the Three$one
member and is used as a mixin in the Two
structure. Since smithy does not allow mixins to be used as targets, we have to create a separate mixin shape, OneMixin
which is used as a mixin for One
which is ultimately what we use for the target in Three
.
- One of the parents is a document rather than a structure
OpenAPI:
openapi: '3.0.'
info:
title: doc
version: 1.0.0
paths: {}
components:
schemas:
One:
type: object
properties: {}
Two:
type: object
properties:
two:
type: string
Three:
type: object
allOf:
- $ref: "#/components/schemas/One"
- $ref: "#/components/schemas/Two"
Smithy:
document One
structure Two {
two: String
}
document Three
In this case, no mixins are created since none are ultimately used. Since One
is translated to a document, Three
must also be a document since it has One
as a parent shape. As such, Two
is never used as a mixin.
The majority of oneOf
schemas in OpenAPI represent untagged unions.
As such, they will be tagged with the alloy#untagged
trait. There are two exceptions to this: tagged unions and discriminated
unions, shown in later examples.
OpenAPI:
openapi: '3.0.'
info:
title: doc
version: 1.0.0
paths: {}
components:
schemas:
Cat:
type: object
properties:
name:
type: string
Dog:
type: object
properties:
breed:
type: string
TestUnion:
oneOf:
- $ref: '#/components/schemas/Cat'
- $ref: '#/components/schemas/Dog'
Smithy:
use alloy#untagged
structure Cat {
name: String
}
structure Dog {
breed: String
}
@untagged
union TestUnion {
Cat: Cat,
Dog: Dog
}
Smithy Translate will convert a oneOf
to a tagged union IF
each of the branches of the oneOf
targets a structure where
each of those structures contains a single required property.
Note that unions in smithy are tagged by default, so there is
no trait annotation required here.
OpenAPI:
openapi: '3.0.'
info:
title: doc
version: 1.0.0
paths: {}
components:
schemas:
Number:
type: object
properties:
num:
type: integer
required:
- num
Text:
type: object
properties:
txt:
type: string
required:
- txt
TestUnion:
oneOf:
- $ref: '#/components/schemas/Number'
- $ref: '#/components/schemas/Text'
Smithy:
structure Number {
@required
num: Integer,
}
structure Text {
@required
txt: String,
}
union TestUnion {
num: Integer,
txt: String
}
Although TestUnion
is a tagged union that can be represented by directly
targeting the Integer
and String
types, Text
and Number
are still rendered.
This is because they are top-level schemas and could be used elsewhere.
A oneOf
will be converted to a discriminated union IF it
contains the discriminator
field. Discriminated unions in
Smithy will be denoted using the alloy#discriminated
trait. The discriminated trait contains the name of the property
that is used as the discriminator.
OpenAPI:
openapi: '3.0.'
info:
title: doc
version: 1.0.0
paths: {}
components:
schemas:
Cat:
type: object
properties:
name:
type: string
pet_type:
type: string
Dog:
type: object
properties:
breed:
type: string
pet_type:
type: string
TestUnion:
oneOf:
- $ref: '#/components/schemas/Cat'
- $ref: '#/components/schemas/Dog'
discriminator:
propertyName: pet_type
Smithy:
use alloy#discriminated
structure Cat {
name: String,
}
structure Dog {
breed: String,
}
@discriminated("pet_type")
union TestUnion {
Cat: Cat,
Dog: Dog
}
OpenAPI:
openapi: '3.0.'
info:
title: doc
version: 1.0.0
paths: {}
components:
schemas:
StringArray:
type: array
items:
type: string
Smithy:
list StringArray {
member: String
}
OpenAPI:
openapi: '3.0.'
info:
title: doc
version: 1.0.0
paths: {}
components:
schemas:
StringSet:
type: array
items:
type: string
uniqueItems: true
Smithy:
@uniqueItems
list StringSet {
member: String
}
OpenAPI:
openapi: '3.0.'
info:
title: doc
version: 1.0.0
paths: {}
components:
schemas:
StringStringMap:
type: object
additionalProperties:
type: string
Smithy:
map StringStringMap {
key: String,
value: String
}
Enums can be translated to either Smithy V1 or V2 syntax. You can control this using the useEnumTraitSyntax
CLI flag.
OpenAPI:
openapi: '3.0.'
info:
title: doc
version: 1.0.0
paths: {}
components:
schemas:
Color:
type: string
enum:
- red
- green
- blue
Smithy:
enum Color {
red
green
blue
}
Or if using the useEnumTraitSyntax
flag:
@enum([
{value: "red"},
{value: "green"},
{value: "blue"}
])
string Color
OpenAPI:
openapi: '3.0.'
info:
title: doc
version: 1.0.0
paths: {}
components:
schemas:
MyString:
type: string
pattern: '^\d{3}-\d{2}-\d{4}$'
Smithy:
@pattern("^\\d{3}-\\d{2}-\\d{4}$")
string MyString
Note that length
, range
, and sensitive
traits are also supported,
as indicated in the primitives table above.
OpenAPI:
openapi: '3.0.'
info:
title: doc
version: 1.0.0
paths:
/test:
post:
operationId: testOperationId
requestBody:
required: true
content:
application/json:
schema:
$ref: '#/components/schemas/ObjectIn'
responses:
'200':
description: test
content:
application/json:
schema:
$ref: '#/components/schemas/ObjectOut'
components:
schemas:
ObjectIn:
type: object
properties:
s:
type: string
required:
- s
ObjectOut:
type: object
properties:
sNum:
type: integer
If provided, such as above, the operationId
will be used
to inform the naming of the operation and the various shapes it
contains.
Smithy:
use smithytranslate#contentType
service FooService {
operations: [TestOperationId]
}
@http(method: "POST", uri: "/test", code: 200)
operation TestOperationId {
input: TestOperationIdInput,
output: TestOperationId200
}
structure ObjectIn {
@required
s: String
}
structure ObjectOut {
sNum: Integer
}
structure TestOperationId200 {
@httpPayload
@required
@contentType("application/json")
body: ObjectOut
}
structure TestOperationIdInput {
@httpPayload
@required
@contentType("application/json")
body: ObjectIn
}
OpenAPI:
openapi: '3.0.'
info:
title: doc
version: 1.0.0
paths:
/test:
get:
operationId: testOperationId
responses:
'200':
description: test
content:
application/json:
schema:
$ref: '#/components/schemas/Object'
'404':
description: test
content:
application/json:
schema:
type: object
properties:
message:
type: string
components:
schemas:
Object:
type: object
properties:
s:
type: string
required:
- s
Smithy:
use smithytranslate#contentType
service FooService {
operations: [TestOperationId]
}
@http(method: "GET", uri: "/test", code: 200)
operation TestOperationId {
input: Unit,
output: TestOperationId200,
errors: [TestOperationId404]
}
structure Object {
@required
s: String
}
@error("client")
@httpError(404)
structure TestOperationId404 {
@httpPayload
@required
@contentType("application/json")
body: Body
}
structure Body {
message: String
}
structure TestOperationId200 {
@httpPayload
@required
@contentType("application/json")
body: Object
}
OpenAPI:
openapi: '3.0.'
info:
title: doc
version: 1.0.0
paths:
/test:
get:
operationId: testOperationId
parameters:
- in: header
name: X-username
schema:
type: string
responses:
'200':
description: test
headers:
X-RateLimit-Limit:
schema:
type: integer
content:
application/json:
schema:
$ref: '#/components/schemas/Object'
components:
schemas:
Object:
type: object
properties:
s:
type: string
required:
- s
Smithy:
use smithytranslate#contentType
service FooService {
operations: [TestOperationId]
}
@http(method: "GET", uri: "/test", code: 200)
operation TestOperationId {
input: TestOperationIdInput,
output: TestOperationId200
}
structure TestOperationIdInput {
@httpHeader("X-username")
X_username: String
}
structure Object {
@required
s: String,
}
structure TestOperationId200 {
@httpPayload
@required
@contentType("application/json")
body: Object,
@httpHeader("X-RateLimit-Limit")
X_RateLimit_Limit: Integer
}
Operations in OpenAPI may contain more than one
content type. This is represented in smithy using a
union
with a special contentTypeDiscriminated
trait.
This trait indicates that the members of the union are
discriminated from one another using the Content-Type
header. Each member of the union is annotated with the contentType
trait. This trait indicates which content type refers to each
specific branch of the union.
OpenAPI:
openapi: '3.0.'
info:
title: doc
version: 1.0.0
paths:
/test:
post:
operationId: testOperationId
requestBody:
required: true
content:
application/octet-stream:
schema:
type: string
format: binary
application/json:
schema:
type: object
properties:
s:
type: string
responses:
'200':
description: test
content:
application/octet-stream:
schema:
type: string
format: binary
application/json:
schema:
type: object
properties:
s:
type: string
Smithy:
use smithytranslate#contentTypeDiscriminated
use smithytranslate#contentType
service FooService {
operations: [TestOperationId]
}
@http(method: "POST", uri: "/test", code: 200)
operation TestOperationId {
input: TestOperationIdInput,
output: TestOperationId200
}
structure TestOperationIdInput {
@httpPayload
@required
body: TestOperationIdInputBody
}
structure TestOperationId200 {
@httpPayload
@required
body: TestOperationId200Body
}
@contentTypeDiscriminated
union TestOperationId200Body {
@contentType("application/octet-stream")
applicationOctetStream: Blob,
@contentType("application/json")
applicationJson: TestOperationId200BodyApplicationJson
}
structure TestOperationId200BodyApplicationJson {
s: String
}
@contentTypeDiscriminated
union TestOperationIdInputBody {
@contentType("application/octet-stream")
applicationOctetStream: Blob,
@contentType("application/json")
applicationJson: TestOperationIdInputBodyApplicationJson
}
structure TestOperationIdInputBodyApplicationJson {
s: String
}
OpenAPI extensions are preserved in the output
Smithy model through the use of the openapiExtensions
trait.
OpenAPI:
openapi: '3.0.'
info:
title: test
version: '1.0'
paths: {}
components:
schemas:
MyString:
type: string
x-float: 1.0
x-string: foo
x-int: 1
x-array: [1, 2, 3]
x-null: null
x-obj:
a: 1
b: 2
Smithy:
use alloy.openapi#openapiExtensions
@openapiExtensions(
"x-float": 1.0,
"x-array": [1, 2, 3],
"x-string": "foo",
"x-int": 1,
"x-null": null,
"x-obj": {
a: 1,
b: 2
}
)
string MyString
> smithytranslate json-schema-to-smithy --help
Usage: smithytranslate json-schema-to-smithy --input <path> [--input <path>]... [--verboseNames] [--failOnValidationErrors] [--useEnumTraitSyntax] [--outputJson] <directory>
Take Json Schema specs as input and produce Smithy files as output.
Options and flags:
--help
Display this help text.
--input <path>, -i <path>
input source files
--verbose-names
If set, names of shapes not be simplified and will be as verbose as possible
--validate-input
If set, abort the conversion if any input specs contains a validation error
--validate-output
If set, abort the conversion if any produced smithy spec contains a validation error
--enum-trait-syntax
output enum types with the smithy v1 enum trait (deprecated) syntax
--json-output
changes output format to be json representations of the smithy models
Run smithytranslate json-schema-to-smithy --help
for all usage information.
Most of the functionality of the OpenAPI => Smithy
conversion is the same for the JSON Schema => Smithy
one. As such, here
we will outline any differences that exist. Everything else is the same.
Default values from JSON Schema will be captured in the smithy.api#default
trait.
JSON Schema:
{
"$id": "test.json",
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "Person",
"type": "object",
"properties": {
"firstName": {
"type": "string",
"default": "Sally"
}
}
}
Smithy:
structure Person {
@default("Sally")
firstName: String
}
JSON Schemas allows for declaring types such as ["string", "null"]
. This type declaration
on a required field means that the value cannot be omitted from the JSON
payload entirely,
but may be set to null
. For example:
JSON Schema:
{
"$id": "test.json",
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "Foo",
"type": "object",
"properties": {
"bar": {
"type": ["string", "null"]
}
},
"required": ["bar"]
}
Smithy:
use alloy#nullable
structure Foo {
@required
@nullable
bar: String
}
In most protocols, there is likely no difference between an optional field and a nullable optional field. Similarly, some protocols may not allow for required fields to be nullable. These considerations are left up to the protocol itself.
JSON Schema doesn't provide a first-class type for defining maps. As such, we translate a commonly-used
convention into map types when encountered. When patternProperties
is set to have a single entry, .*
,
we translate that to a smithy map type.
JSON Schema:
{
"$id": "test.json",
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "TestMap",
"type": "object",
"patternProperties": {
".*": {
"type": "string"
}
}
}
Smithy:
map TestMap {
key: String,
value: String
}
> smithytranslate smithy-to-proto --help
Usage: smithytranslate smithy-to-proto --input <path> [--input <path>]... [--dependency <string>]... [--repository <string>]... <directory>
Take Smithy definitions as input and produce Proto files as output.
Options and flags:
--help
Display this help text.
--input <path>, -i <path>
input source files
--dependency <string>
Dependencies that contains Smithy definitions.
--repository <string>
Specify repositories to fetch dependencies from.
Run smithytranslate smithy-to-proto --help
for more usage information.
The design of the smithy to protobuf translation follows the semantics defined in the alloy specification.
Individual protobuf definitions file (.proto
) can contain options. We support this feature using Smithy's metadata attribute.
There are a few importing things to notice
- All options are defined under the metadata key
proto_options
- The value is an array. This is because Smithy will concatenate the arrays if the model contains multiple entries
- Each entry of the array is an object where the keys are the namespace and the values are objects that represent the options
- Entries for other namespaces are ignored (for example,
demo
in the example below) - The object that represents an option can only use
String
as value (see the example below). More detail below.
We used a String
to represent the option such as "true"
for a boolean and "\"demo\""
for a String because it's the simplest approach to cover all use cases supported by protoc
. protoc
supports simple types like you'd expect: bool
, int
, float
and string
. But it also supports identifier
which are a reference to some value that protoc
knows about. For example: option optimize_for = CODE_SIZE;
. Using a String
for the value allows us to model this, while keeping thing simple. It allows prevent the users from trying to use Array
s or Object
as value for options.
The following is an example:
Smithy:
$version: "2"
metadata "proto_options" = [{
"foo": {
"java_multiple_files": "true",
"java_package": "\"foo.pkg\""
},
"demo": {
"java_multiple_files": "true"
}
}]
namespace foo
structure Foo {
value: String
}
Proto:
syntax = "proto3";
option java_multiple_files = true;
option java_package = "foo.pkg";
package foo;
message Foo {
string value = 1;
}