Remote Procedure Call (RPC) is a protocol that one program can use to request a service from a program located in another computer in a network without having to understand network details. (A procedure call is also sometimes known as a function call or a subroutine call.) RPC uses the client/server model.
The Apache Thrift software framework, for scalable cross-language services development, combines a software stack with a code generation engine to build services that work efficiently and seamlessly between C++, Java, Python, PHP, Ruby, Erlang, Perl, Haskell, C#, Cocoa, JavaScript, Node.js, Smalltalk, OCaml and Delphi and other languages.
Thrift Network Stack
- Server - Single-threaded, event-driven etc.
- Processor - Compiler generated.
- Protocol - JSON, compact etc.
- Transport - Raw TCP, HTTP etc.
Protocol Layer
The protocol layer provides serialization and deserialization. Thrift supports the following protocols:
- TBinaryProtocol - A straight-forward binary format encoding numeric values as binary, rather than converting to text.
- TCompactProtocol - Very efficient, dense encoding of data (See details below).
- TDenseProtocol - Similar to TCompactProtocol but strips off the meta information from what is transmitted, and adds it back in at the receiver. TDenseProtocol is still experimental and not yet available in the Java implementation.
- TJSONProtocol - Uses JSON for encoding of data.
- TSimpleJSONProtocol - A write-only protocol using JSON. Suitable for parsing by scripting languages.
- TDebugProtocol - Uses a human-readable text format to aid in debugging.
Tranport Layer
The transport layer is responsible for reading from and writing to the wire. Thrift supports the following:
- TSocket - Uses blocking socket I/O for transport.
- TFramedTransport - Sends data in frames, where each frame is preceded by a length. This transport is required when using a non-blocking server.
- TFileTransport - This transport writes to a file. While this transport is not included with the Java implementation, it should be simple enough to implement.
- TMemoryTransport - Uses memory for I/O. The Java implementation uses a simple ByteArrayOutputStream internally.
- TZlibTransport - Performs compression using zlib. Used in conjunction with another transport. Not available in the Java implementation.
Install Thrift
Install Apache Thrift on OS X via Homebrew and validate Thrift version:
$ brew install thrift && thrift -version
Thrift IDL
The Thrift interface definition language (IDL) allows for the definition of Thrift Types. A Thrift IDL file is processed by the Thrift code generator to produce code for the various target languages to support the defined structs and services in the IDL file. The first thing to know about are types. The available types in Thrift are:
- bool - Boolean, one byte
- i8 (byte) - Signed 8-bit integer
- i16 - Signed 16-bit integer
- i32 - Signed 32-bit integer
- i64 - Signed 64-bit integer
- double - 64-bit floating point value
- string - String
- binary - Blob (byte array)
- map<t1,t2> - Map from one type to another
- list<t1> - Ordered list of one type
- set<t1> - Set of unique elements of one type
Create mythrift.thrift file:
/**
* Thrift files can namespace, package, or prefix their output in various
* target languages.
*/
namespace go mythrift.demo
namespace php mythrift.demo
/**
* Structs are the basic complex data structures. They are comprised of fields
* which each have an integer identifier, a type, a symbolic name, and an
* optional default value.
*
* Fields can be declared "optional", which ensures they will not be included
* in the serialized output if they aren't set. Note that this requires some
* manual management in some languages.
*/
struct Article{
1: i32 id,
2: string title,
3: string content,
4: string author,
}
const map<string,string> MAPCONSTANT = {'hello':'world', 'goodnight':'moon'}
service myThrift {
list<string> CallBack(1:i64 callTime, 2:string name, 3:map<string, string> paramMap),
void put(1: Article newArticle),
}
Compile IDL File:
$ thrift -r --gen go mythrift.thrift $ thrift -r --gen php mythrift.thrift $ thrift -r --gen php:server mythrift.thrift
After execute above command will generate a gen-go folder, and copy gen-go/mythrift folder to $GOPATH.
Get Go Thrift package:
$ go get github.com/apache/thrift/lib/go/thrift
Go Implement Thrift Server
Create main.go file in thrift-server folder:
package main
import (
"fmt"
"os"
"time"
"git.apache.org/thrift.git/lib/go/thrift"
"mythrift/demo" // Import interface package generated by Thrift
)
const (
NetworkAddr = "127.0.0.1:9090"
)
type mythriftThrift struct{}
func (this *mythriftThrift) CallBack(callTime int64, name string, paramMap map[string]string) (r []string, err error) {
fmt.Println("-->from client Call:", time.Unix(callTime, 0).Format("2006-01-02 15:04:05"), name, paramMap)
r = append(r, "key:"+paramMap["a"]+" value:"+paramMap["b"])
return
}
func (this *mythriftThrift) Put(s *demo.Article) (err error) {
fmt.Printf("Article--->id: %d\tTitle:%s\tContent:%s\tAuthor:%s\n", s.ID, s.Title, s.Content, s.Author)
return nil
}
func main() {
transportFactory := thrift.NewTFramedTransportFactory(thrift.NewTTransportFactory())
protocolFactory := thrift.NewTBinaryProtocolFactoryDefault()
serverTransport, err := thrift.NewTServerSocket(NetworkAddr)
if err != nil {
fmt.Println("Error!", err)
os.Exit(1)
}
handler := &mythriftThrift{}
processor := demo.NewMyThriftProcessor(handler)
server := thrift.NewTSimpleServer4(processor, serverTransport, transportFactory, protocolFactory)
fmt.Println("thrift server in", NetworkAddr)
server.Serve()
}
Go Implement Thrift Client
Create main.go file in thrift-client folder:
package main
import (
"fmt"
"net"
"os"
"strconv"
"time"
"git.apache.org/thrift.git/lib/go/thrift"
"mythrift/demo"
)
const (
HOST = "127.0.0.1"
PORT = "9090"
)
func main() {
startTime := currentTimeMillis()
transportFactory := thrift.NewTFramedTransportFactory(thrift.NewTTransportFactory())
protocolFactory := thrift.NewTBinaryProtocolFactoryDefault()
transport, err := thrift.NewTSocket(net.JoinHostPort(HOST, PORT))
if err != nil {
fmt.Fprintln(os.Stderr, "error resolving address:", err)
os.Exit(1)
}
useTransport := transportFactory.GetTransport(transport)
client := demo.NewMyThriftClientFactory(useTransport, protocolFactory)
if err := transport.Open(); err != nil {
fmt.Fprintln(os.Stderr, "Error opening socket to "+HOST+":"+PORT, " ", err)
os.Exit(1)
}
defer transport.Close()
for i := 0; i < 10; i++ {
paramMap := make(map[string]string)
paramMap["a"] = "mythrift.demo"
paramMap["b"] = "test" + strconv.Itoa(i+1)
r1, _ := client.CallBack(time.Now().Unix(), "Go client", paramMap)
fmt.Println("Go client call->", r1)
}
model := demo.Article{1, "Send from Go Thrift Client", "Hello World!", "Go"}
client.Put(&model)
endTime := currentTimeMillis()
fmt.Printf("The call took:%d-%d=%d Millis \n", endTime, startTime, (endTime - startTime))
}
func currentTimeMillis() int64 {
return time.Now().UnixNano() / 1000000
}
PHP Implement Thrift Client
Clone git@github.com:apache/thrift.git and import thrift/lib/php/lib in PHP script, create phpthrift-client folder copy gen-php folder into it. Create main.php in phpthrift-client folder:
<?php
/**
* Thrift RPC - PHP Client
*/
namespace mythrift\demo;
error_reporting(E_ALL);
$startTime = getMillisecond();
$ROOT_DIR = realpath(dirname(__FILE__).'/');
$GEN_DIR = realpath(dirname(__FILE__).'/') . '/gen-php';
require_once $ROOT_DIR . '/Thrift/ClassLoader/ThriftClassLoader.php';
use Thrift\ClassLoader\ThriftClassLoader;
use Thrift\Protocol\TBinaryProtocol;
use Thrift\Transport\TSocket;
use Thrift\Transport\TSocketPool;
use Thrift\Transport\TFramedTransport;
use Thrift\Transport\TBufferedTransport;
$loader = new ThriftClassLoader();
$loader->registerNamespace('Thrift',$ROOT_DIR);
$loader->registerDefinition('mythrift\demo', $GEN_DIR);
$loader->register();
$thriftHost = '127.0.0.1';
$thriftPort = 9090;
$socket = new TSocket($thriftHost,$thriftPort);
$socket->setSendTimeout(10000); // Sets the send timeout.
$socket->setRecvTimeout(20000); // Sets the receive timeout.
$transport = new TFramedTransport($socket);
$protocol = new TBinaryProtocol($transport);
$client = new \mythrift\demo\myThriftClient($protocol);
$transport->open();
$socket->setDebug(TRUE);
for ($i=0; $i < 10; $i++) {
$item = array();
$item['a'] = 'mythrift.demo';
$item['b'] = 'test' + $i;
$result = $client->CallBack(time(), 'PHP Client', $item);
echo 'PHP Client Call->', implode('', $result), PHP_EOL;
}
$s = new \mythrift\demo\Article();
$s->id = 1;
$s->title = 'Send from PHP Thrift Client';
$s->content = 'Hello World!';
$s->author = 'PHP';
$client->put($s);
$endTime = getMillisecond();
echo 'The call took:', $endTime, '-', $startTime, '=', ($endTime - $startTime), ' Millis', PHP_EOL;
function getMillisecond() {
list($t1, $t2) = explode(' ', microtime());
return (float)sprintf('%.0f', (floatval($t1) + floatval($t2)) * 1000);
}
$transport->close();
Build and Testing
Build and run thrift-server:
$ go build thrift-server && ./thrift-server thrift server in 127.0.0.1:9090
Build and run thrift-client:
$ go build thrift-client && ./thrift-client Go client call-> [key:mythrift.demo value:test1] Go client call-> [key:mythrift.demo value:test2] Go client call-> [key:mythrift.demo value:test3] Go client call-> [key:mythrift.demo value:test4] Go client call-> [key:mythrift.demo value:test5] Go client call-> [key:mythrift.demo value:test6] Go client call-> [key:mythrift.demo value:test7] Go client call-> [key:mythrift.demo value:test8] Go client call-> [key:mythrift.demo value:test9] Go client call-> [key:mythrift.demo value:test10] The call took:1467012021504-1467012021428=76 Millis
Thirft server output:
-->from client Call: 2016-06-27 15:20:21 Go client map[a:mythrift.demo b:test1] -->from client Call: 2016-06-27 15:20:21 Go client map[b:test2 a:mythrift.demo] -->from client Call: 2016-06-27 15:20:21 Go client map[b:test3 a:mythrift.demo] -->from client Call: 2016-06-27 15:20:21 Go client map[a:mythrift.demo b:test4] -->from client Call: 2016-06-27 15:20:21 Go client map[b:test5 a:mythrift.demo] -->from client Call: 2016-06-27 15:20:21 Go client map[a:mythrift.demo b:test6] -->from client Call: 2016-06-27 15:20:21 Go client map[b:test7 a:mythrift.demo] -->from client Call: 2016-06-27 15:20:21 Go client map[b:test8 a:mythrift.demo] -->from client Call: 2016-06-27 15:20:21 Go client map[a:mythrift.demo b:test9] -->from client Call: 2016-06-27 15:20:21 Go client map[a:mythrift.demo b:test10] Article--->id: 1 Title:Send from Go Thrift Client Content:Hello World! Author:Go
Run phpthrift-client/main.php:
$ php main.php PHP Client Call->key:mythrift.demo value:0 PHP Client Call->key:mythrift.demo value:1 PHP Client Call->key:mythrift.demo value:2 PHP Client Call->key:mythrift.demo value:3 PHP Client Call->key:mythrift.demo value:4 PHP Client Call->key:mythrift.demo value:5 PHP Client Call->key:mythrift.demo value:6 PHP Client Call->key:mythrift.demo value:7 PHP Client Call->key:mythrift.demo value:8 PHP Client Call->key:mythrift.demo value:9 The call took:1467018064708-1467018064646=62 Millis
Thirft server output:
-->from client Call: 2016-06-27 17:01:04 PHP Client map[a:mythrift.demo b:0] -->from client Call: 2016-06-27 17:01:04 PHP Client map[a:mythrift.demo b:1] -->from client Call: 2016-06-27 17:01:04 PHP Client map[a:mythrift.demo b:2] -->from client Call: 2016-06-27 17:01:04 PHP Client map[a:mythrift.demo b:3] -->from client Call: 2016-06-27 17:01:04 PHP Client map[a:mythrift.demo b:4] -->from client Call: 2016-06-27 17:01:04 PHP Client map[a:mythrift.demo b:5] -->from client Call: 2016-06-27 17:01:04 PHP Client map[a:mythrift.demo b:6] -->from client Call: 2016-06-27 17:01:04 PHP Client map[a:mythrift.demo b:7] -->from client Call: 2016-06-27 17:01:04 PHP Client map[a:mythrift.demo b:8] -->from client Call: 2016-06-27 17:01:04 PHP Client map[a:mythrift.demo b:9] Article--->id: 1 Title:Send from PHP Thrift Client Content:Hello World! Author:PHP