CODEWITHSUNDEEP
thriftistioenvoyproxy
pnovotnak
pnovotnak

Reputation: 4581

Thrift Rate Limiting with Envoy + Istio

I'm trying to add very basic rls[1] support to Istio for Thrift protocol stacks[2] by allowing a user to provide an external ratelimit service[3] as an environment variable to Pilot. I've run into an issue where the config seems to be rejected by the client.

Here is an isolated and simplified version of my Istio control plane code:

import (
    "github.com/envoyproxy/go-control-plane/envoy/api/v2/core"
    "istio.io/istio/pilot/pkg/model"
    "istio.io/istio/pilot/pkg/networking/util"
    ratelimit "github.com/envoyproxy/go-control-plane/envoy/config/ratelimit/v2"
    thrift_proxy "github.com/envoyproxy/go-control-plane/envoy/config/filter/network/thrift_proxy/v2alpha1"
    thrift_ratelimit "github.com/envoyproxy/go-control-plane/envoy/config/filter/thrift/rate_limit/v2alpha1"
)

// thriftListenerOpts are options for a Thrift listener
type thriftListenerOpts struct {
    // stat prefix for the thrift connection manager
    // DO not set this field. Will be overridden by buildCompleteFilterChain
    statPrefix  string
    transport   int
    protocol    int
    routeConfig *thrift_proxy.RouteConfiguration
}

func buildRatelimtedThriftProxy(node *model.Proxy, thriftOpts *thriftListenerOpts, ratelimitServiceUri string) *thrift_proxy.ThriftProxy {
    var ratelimitService *thrift_ratelimit.RateLimit
    var ratelimitFilter  *thrift_proxy.ThriftFilter
    var proxy *thrift_proxy.ThriftProxy

    proxy = &thrift_proxy.ThriftProxy{
        StatPrefix:  thriftOpts.statPrefix,
        Transport:   thrift_proxy.HEADER,
        Protocol:    thrift_proxy.BINARY,
        RouteConfig: thriftOpts.routeConfig,
    }

    ratelimitFilter = &thrift_proxy.ThriftFilter{
        Name: "config.filter.thrift.rate_limit.v2alpha1.RateLimit",
    }

    ratelimitService = &thrift_ratelimit.RateLimit{
        RateLimitService: &ratelimit.RateLimitServiceConfig{
            GrpcService: &core.GrpcService{
                TargetSpecifier: &core.GrpcService_GoogleGrpc_{
                    GoogleGrpc: &core.GrpcService_GoogleGrpc{
                        TargetUri: ratelimitServiceUri,
                    },
                },
            },
        },
    }

    if util.IsXDSMarshalingToAnyEnabled(node) {
        ratelimitFilter.ConfigType = &thrift_proxy.ThriftFilter_TypedConfig{TypedConfig: util.MessageToAny(ratelimitService)}
    } else {
        ratelimitFilter.ConfigType = &thrift_proxy.ThriftFilter_Config{Config: util.MessageToStruct(ratelimitService)}
    }

    proxy.ThriftFilters = append(proxy.ThriftFilters, ratelimitFilter)

    return proxy
}

When checking the /config_dump endpoint of Envoy's admin interface, when the ratelimit configuration is not supplied, the proxy is present and configured correctly. As soon as I try to add the ratelimit config, it disappears entirely.

What am I doing wrong?

  1. Source code that defines Thrift ratelimit api: https://github.com/envoyproxy/envoy/blob/master/api/envoy/config/filter/thrift/rate_limit/v2alpha1/rate_limit.proto#L9
  2. See official docs here: https://www.envoyproxy.io/docs/envoy/latest/api-v2/config/filter/thrift/rate_limit/v2alpha1/rate_limit.proto
  3. Reference implementation here: https://github.com/lyft/ratelimit

Upvotes: 1

Views: 528

Answers (2)

huabing zhao
huabing zhao

Reputation: 41

The build-in support of Thrift and other layer-7 protocols in Istio is limited or none. Consider Aeraki if you want full capabilities that have already been supported at the EnvoyProxy side. Aeraki can help to support Thrift, or other layer-7 protocols, in an Istio service mesh, including traffic management, such as version-based routing and traffic splitting, and service metrics.

Demo: http://aeraki.zhaohuabing.com:3000/d/pgz7wp-Gz/aeraki-demo?orgId=1&refresh=10s&kiosk https://www.youtube.com/watch?v=vrjp-Yg3Leg

Upvotes: 0

pnovotnak
pnovotnak

Reputation: 4581

Turns out I had my GrpcService configured incorrectly. It was missing the StatPrefix field. I was able to figure this out by looking at the envoy logs. I was also able to check for this in my code by running ratelimitService.Validate() which validates recursively.

Upvotes: 1

Related Questions