Thrift

Apache Thrift is a popular framework for working with data types and service interfaces. It uses an Interface Definition Language (IDL) to define these types and interfaces. There are tools to generate code in "real" programming languages from Thrift IDL files. Two programs, perhaps in different programming languages, should be able to communicate over Thrift interfaces by using this generated code.

Pants knows Thrift. For each Thrift file you use, your codebase has some BUILD targets that represent "real" code generated from IDL code. You can write code in your favorite language that imports the generated code. To make the import work, your code's BUILD target depends on the appropriate Thrift BUILD target.

Generating Code

You have some Thrift; you want to use it from your "regular" programming language. Normally, to make, e.g., Java code usable, you set up a java_library target with sources *.java and then depend on that target; Thrift works similarly, but you use a different target type that generates Java code from *.thrift. You can define Java, Python, or Scala library targets whose code is Thrift-generated by setting up lang_thrift_library targets. (Scala is tricky; you use a java_thrift_library with carefully-chosen parameters.) Other targets can depend on a lang_thrift_library and their code can then import the generated code.

Target Example

This example sets up a java_thrift_library target; its source is Thrift; it generates Java.

# Target defined in src/thrift/com/twitter/mybird/BUILD:
java_thrift_library(
  name='mybird',
  # Specify dependencies for thrift IDL file includes.
  dependencies=[
    'src/thrift/com/twitter/otherbird',
  ],
  sources=['*.thrift']
)

Pants knows that before it compiles such a target, it must first generate Java code from the Thrift IDL files. Users can depend on this target like any other internal target. In this case, users would add a dependency on 'src/thrift/com/twitter/mybird'.

One lang_thrift_library can depend on another; use this if one of your Thrift files includes a Thrift file that lives in another target.

Configuring

Here are some popular *_thrift_library configurations:

Java

Use Apache Thrift compiler (the default):

java_thrift_library(...)

...or Scrooge:

java_thrift_library(
  compiler='scrooge')

Python

python_thrift_library(...)

Scala

java_thrift_library(  # Yes, a "java" library to generate Scala
  compiler='scrooge', # default compiler does not gen Scala; Scrooge does
  language='scala',
  # maybe set compiler_args
)

Android

Scrooge compiler can now generate java code with a limited number of getters and setters which is more suitable for Android applications which have a 65K limit on the number of methods.

java_thrift_library(  # Yes, a "java" library to generate Scala
  compiler='scrooge', # default compiler does not gen android; Scrooge does
  language='android',
)

Thrift Example

Let's look at some sample code that puts all of this together.

• Thrift IDL code (.thrift files)
• BUILD targets for the Thrift IDL code
• Java code that imports code generated from Thrift
• BUILD target dependencies that allow that import

Thrift IDL

Our example uses two Thrift files, one of which includes the other. They look pretty ordinary. The included Thrift, examples/src/thrift/org/pantsbuild/example/distance/distance.thrift, is regular Thrift:

// Copyright 2014 Pants project contributors (see CONTRIBUTORS.md).
// Licensed under the Apache License, Version 2.0 (see LICENSE).

namespace java org.pantsbuild.example.distance.thriftjava
namespace py org.pantsbuild.example.distance

/*
 * Structure for expressing distance measures: 8mm, 12 parsecs, etc.
 * Not so useful on its own.
 

The includeing Thrift, examples/src/thrift/org/pantsbuild/example/precipitation/precipitation.thrift, also looks ordinary. (The include path is a little tricky: it's based on source roots. Thus, if your source tree has more than one root foo and bar and has Thrift in both, code in foo can include code from bar without mentioning bar in the include path.):

// Copyright 2014 Pants project contributors (see CONTRIBUTORS.md).
// Licensed under the Apache License, Version 2.0 (see LICENSE).

namespace java org.pantsbuild.example.precipitation.thriftjava
namespace py org.pantsbuild.example.precipitation

include "org/pantsbuild/example/distance/distance.thrift"

/*
 * Structure for recording weather events, e.g., 8mm of rain.
 

BUILD Targets

In a BUILD file, we use a java_thrift_library or python_thrift_library to generate "real" code from Thrift. Our example just uses Java; thus, the BUILD file for distance.thrift looks like

java_thrift_library(name='distance-java',
  sources=['distance.thrift'],
  provides = artifact(org='org.pantsbuild.example',
                      name='distance-thrift-java',
                      repo=public),
)

python_thrift_library(name='distance-python',
  sources=['distance.thrift'],
  provides=setup_py(
    name='pantsbuild.pants.distance-thrift-python',
    version='0.0.1',
  )
)

python_thrift_library(name='unexported-distance-python',
  sources=['unexported_distance.thrift'],
)

Notice the target type is java_thrift_library, and this target staked its claim to our distance thrift IDL file. JVM library targets (e.g.: java_library, scala_library) that depend on this target will simply see generated code from the IDL. Since no additional options are specified we use the defaults; however, if we need more control over how code is generated we control that through arguments provided by java_thrift_library.

note

While the name java_thrift_library might make you think it generates Java, it can also generate other target languages via the language parameter (scala for example). For Python code, however, use python_thrift_library.

As with "regular" languages, for one target's code to include another's, a target should have the other in its dependencies. Thus, to allow precipitation.thrift to depend on distance.thrift, we set up .../precipitation/BUILD like so:

java_thrift_library(name='precipitation-java',
  sources=['precipitation.thrift'],
  dependencies=[
    'examples/src/thrift/org/pantsbuild/example/distance:distance-java',
  ],
  provides = artifact(org='org.pantsbuild.example',
                      name='precipitation-thrift-java',
                      repo=public),
)

python_thrift_library(name='precipitation-python',
  sources=['precipitation.thrift'],
  dependencies=[
    'examples/src/thrift/org/pantsbuild/example/distance:distance-python',
  ],
  provides=setup_py(
    name='pantsbuild.pants.precipitation-thrift-python',
    version='0.0.1',
  )
)

python_thrift_library(name='monolithic-precipitation-python',
  sources=['monolithic_precipitation.thrift'],
  dependencies=[
    'examples/src/thrift/org/pantsbuild/example/distance:unexported-distance-python',
  ],
  provides=setup_py(
    name='pantsbuild.pants.monolithic-precipitation-thrift-python',
    version='0.0.1',
  )
)

Using in "Regular" Code

We want to use the Thrift-generated interface from "regular" code. In this Java example, we want to import the generated code. In our Java, the import statements use the names from the .thrift files' namespaces:

package org.pantsbuild.example.usethrift;

import org.junit.Test;

import org.pantsbuild.example.distance.thriftjava.Distance;
import org.pantsbuild.example.precipitation.thriftjava.Precipitation;

/ Not testing behavior; we're happy if this compiles. /
public class UseThriftTest {
  @Test
  public void makeItRain() {
    Distance notMuch = new Distance().setNumber(8).setUnit("mm");
    Precipitation sprinkle = new Precipitation().setDistance(notMuch);
  }
}

As usual, for code in one target to use code from another, one target needs to depend on the other. Thus, our Java code's target has the *_thrift_library target whose code it uses in its dependencies:

junit_tests(
  sources=['UseThriftTest.java',],
  dependencies=[
    '3rdparty:junit',
    '3rdparty:thrift',
    'examples/src/thrift/org/pantsbuild/example/distance:distance-java',
    'examples/src/thrift/org/pantsbuild/example/precipitation:precipitation-java',
  ],
)

Publishing

Publishing a lang_thrift_library is like publishing a "regular" library The targets use provides parameters. It might look something like:

java_thrift_library(name='eureka-java',
  sources=['eureka.thrift'],
  dependencies=[
    'src/thrift/org/archimedes/volume:volume-java',
  ],
  language='java',
  provides=artifact(
    org='org.archimedes',
    name='eureka-java',
    repo='BUILD.archimedes:jar-public',
))

java_thrift_library(name='eureka-scala',
  sources=['eureka.thrift'],
  dependencies=[
    'src/thrift/org/archimedes/volume:volume-scala',
  ],
  compiler='scrooge',
  language='scala',
  provides=artifact(
    org='org.archimedes',
    name='eureka-scala',
    repo='BUILD.archimedes:jar-public',
  ))