Thrift Debugging: Problems with my finagle server

Question

I have a simple Thrift Server, implemented in scala with finagle:

    import com.twitter.util.{ Await, Future }
    import com.jakiku.thriftscala.{ RealTimeDatabasePageImpressions, PageImpressions }
    import com.twitter.finagle.Thrift
    import com.twitter.finagle.thrift.ThriftServerFramedCodec
    import com.twitter.finagle.builder.{ ServerBuilder, Server }

    object ThriftServer {

      def main(args: Array[String]) {
        val server = Thrift.serveIface("localhost:9090", new RealTimeDatabasePageImpressions[Future] {
          def getByTrackIdAndDay(trackId: Int, day: Int) = {
            Future(Seq(PageImpressions.apply(123, 3, 4,  3,  2000L, 2000L)))
          }
        })
        Await.ready(server)
      }
    }

This is my Thrift file:

    namespace   java com.jakiku.thriftjava
    #@namespace scala com.jakiku.thriftscala

    typedef i64 long
    typedef i32 int

    struct PageImpressionsSum{
        1: required int         trackId;
        2: required int         day;
        3: required int         hour;
        4: required int         minute;
        5: required string      pageId;
        6: required long        uniqueImpressions;
        7: required long        sumImpressions;
    }

    struct PageImpressions{
        1: required int         trackId;
        2: required int         day;
        3: required int         hour;
        4: required int         minute;
        6: required long        uniqueImpressions;
        7: required long        sumImpressions;
    }

    service RealTimeDatabase_pageImpressions{
      list<PageImpressions> getByTrackIdAndDay(1:int trackId, 2:int day);
    }

    service RealTimeDatabase_pageImpressionsSum {
      list<PageImpressionsSum> getByTrackIdAndDay(1:int trackId, 2:int day);
    }

I generated the scala thrift classes with scrooge. Added thriftlib, scrooge-core, scrooge-generator etc. as dependency. I also added the scrooge sbt plugin:

 addSbtPlugin("com.twitter" % "scrooge-sbt-plugin" % "4.5.0")

I started the server with sudo sbt 'run'. The output of the console:

[info] Running ThriftServer Mai 25, 2016 4:18:39 AM com.twitter.finagle.Init$$anonfun$1 apply$mcV$sp INFORMATION: Finagle version 6.34.0 (rev=44f444f606b10582c2da8d5770b7879ddd961211) built at 20160310-155158

Everything looks fine, at this moment. I tested the server in nodejs. Doesn't work. I also used the thrift python test suite. Every check runs into a timeout.

This is the nodejs client:

    var thrift = require('thrift');
    var Realtime_pageImpressions = require('./gen-nodejs/RealTimeDatabase_pageImpressions.js');
    var ttypes = require('./gen-nodejs/RealTimeDatabase_types.js');


    var connection = thrift.createConnection("localhost", 9090);

    connection.on('error', function(err) {
        console.log(err);
    });

    var client = thrift.createClient(Realtime_pageImpressions, connection);

    client.getByTrackIdAndDay(123124, 4, function(err, response) {
        if (err) {
        console.error(err);
        } else {
        console.log(response);
        }
    });

I really have no idea what i'm doing wrong.

Answer 1

Okay, i fixed it. Two things: @BCG gave me a hint. It's the TFramedTransport. But the Apache Docu is a bit too old.

 var ThriftTransports = require('thrift/transport');
 var ThriftProtocols = require('thrift/protocol');

thrift/transport and thrift/protocol don't exist anymore. You have only to require Thrift. I deleted also the two brackets for the function execution.

    var thrift = require('thrift');
    var Int64 = require('node-int64');
    var Realtime_pageImpressions = require('./gen-nodejs/RealTimeDatabase_pageImpressions.js');
    var ttypes = require('./gen-nodejs/RealTimeDatabase_types.js');

    transport   = thrift.TFramedTransport;
    protocol    = thrift.TBinaryProtocol;

    var connection = thrift.createConnection("localhost", 9090, {
        transport : transport,
        protocol : protocol
    });

    connection.on('error', function(err) {
        console.log(err);
    });

    // Create a Calculator client with the connection
    var client = thrift.createClient(Realtime_pageImpressions, connection);

    client.getByTrackIdAndDay(123124, 4, function(err, response) {
        if (err) {
        console.error(err);
        } else {
        console.log(response);
        console.log(response[0].uniqueImpressions[0]);
        }
    });

Answer 2

Not sure if this is the problem (especially since I'm not familiar with scrooge or finagle) but it looks like you're using a Framed transport on the server and maybe not on the client.

Looking at the node.js Thrift library code, it seems as if TBufferedTransport is the default, and you might want TFramedTransport instead.

On the node.js Thrift tutorial page, they show how you can initialize the connection with overridden options:

var thrift = require('thrift');
var ThriftTransports = require('thrift/transport');
var ThriftProtocols = require('thrift/protocol');
var Calculator = require('./gen-nodejs/Calculator');
var ttypes = require('./gen-nodejs/tutorial_types');

transport = ThriftTransports.TBufferedTransport()
protocol = ThriftProtocols.TBinaryProtocol()

var connection = thrift.createConnection("localhost", 9090, {
  transport : transport,
  protocol : protocol
});

You might try implementing your node.js client as above, except replace with TFramedTransport and see if that helps.