CODEWITHSUNDEEP
prestoconnector
max.ott
max.ott

Reputation: 703

How to use pushdown predicates in Presto connector

I would like to extend Presto's example-http connector to add the relevant predicates to the REST API call made by the connector to minimize data transfer.

I have been spending now quite a few hours looking through the documentation, online posts and the presto source code but I can't figure out which calls are involved and how to go about this.

It's being mentioned in so many places, but I cannot find any simple code samples or internal descriptions. I'm sure I'm missing something obvious here.

Upvotes: 3

Views: 1407

Answers (1)

max.ott
max.ott

Reputation: 703

After downloading the source code and running it in a debugger I found it to be rather straight forward on one hand, while restrictive on the other.

The straight forward part is that we simply have to override isPushdownFilterSupported and pushdownFilter when implementing the ConnectorMetadata interface.

The restrictive part is that the query planner now believes that the connector can deal with any type and combination of table filters. In my case, I would only want to take care of those, the remote API I'm calling is supporting and have Presto take care of the rest.

It appears the Presto team is fully aware of that, as a) the methods are labeled @Experimental and b) by the respective comments This interface can be replaced with a connector optimizer rule once the engine supports these (#12546). This would be clearly the right approach for my use case.

   /**
     * Experimental: if true, the engine will invoke pushdownFilter instead of getTableLayouts.
     *
     * This interface can be replaced with a connector optimizer rule once the engine supports these (#12546).
     */
    @Experimental
    default boolean isPushdownFilterSupported(ConnectorSession session, ConnectorTableHandle tableHandle)
    {
        return false;
    }

    /**
     * Experimental: returns table layout that encapsulates the given filter.
     *
     * This interface can be replaced with a connector optimizer rule once the engine supports these (#12546).
     */
    @Experimental
    default ConnectorPushdownFilterResult pushdownFilter(ConnectorSession session, ConnectorTableHandle tableHandle, RowExpression filter, Optional<ConnectorTableLayoutHandle> currentLayoutHandle)
    {
        throw new UnsupportedOperationException();
    }

Upvotes: 1

Related Questions