CODEWITHSUNDEEP
pythonjsonsocketsnetwork-programmingtornado
Dawid Dave Kosiński
Dawid Dave Kosiński

Reputation: 901

Connect to second server API trough sockets

I'm writing a project in Python that will have two diffrent (separate) apps.

  1. A proxy (normal Python code, no framework)
  2. A database with API to send/receive JSON (written in tornado 4.4 framework)

My project will do something like this:

  1. In the database app you can add filters: 1 - anchor tags which you want to filter out a web page, 2 - URL patterns that you want to give 403 status.
  2. The database app on a specyfic URL (api/send-to-world/filter_type/) serves a JSON with all the filters of a filter_type = filter_type (from above)
  3. While accessing a web page, the proxy first gets the filters by catch a JSON on the URL above and then does the filter stuff
  4. For statistics, the filtered content is send to the database API that receives a JSON (url: /api/receive-from-world/) and saves the content to the database.

To test the project I put the to apps on my localhost:

  1. PROXY on port 8000
  2. Database app on port 900

The project worked fine when the two apps where written thanks to Tornado framework but for personal reasons I want to make the PROXY a litte more low - level -> using sockets.

So to do this I've rewitten the Proxy app to a pure Python code and I got a little problem. The problem is that when I create a socket and I want to receive a JSON from the database API the database app gives me a 404 error (while if I access that URL on the database it works fine). I don't know what can be the problem.

My proxy class with the send_json and socket creation method:

class Proxy(object):
    u"""The PROXY class."""
    (...)

    def get_remote_response(self, host, port, data):
        u"""Method that creates a socket to the remote_address and gets the response.
        :param host: remote_host,
        :param port: remote_port,
        :param data: data to send.
        :return response: the response without the headers.
        """
        response_text = ''
        try:
            remote_sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
            print "Trying to connect to:%s : %s \n\n" % (host, port)
            remote_sock.connect((host, port))
            remote_sock.send(data)
            remote_sock.settimeout(0.5)
            while True:
                try:
                    response = remote_sock.recv(self.buffor_size)

                    if len(response) == 0:
                        break
                except socket.error:
                    break
                response_text += response
        except socket.error, msg:
            print str(msg)
        remote_sock.close()
        response = response_text[response_text.index('\r\n\r\n') + 1:]
        return response

    def get_filters(self, filter_id, **kwargs):
        u"""Method that gets a JSON with filters from the remote server
        :param filter_id: filter id.
        :return filters: list of filters from the remote server.
        """
        filters = list()
        remote_host = '127.0.0.1'
        remote_port = 9000
        request_url = 'http://%s:%s/api/send-to-world/%s/' % (remote_host, str(remote_port), filter_id)
        first_line_of_request = 'GET ' + request_url + ' HTTP/1.1\r\n'
        headers = {
            'Host': remote_host,
            'Content-Type': 'application/json',
            'Accept': 'application/json'
        }
        request = first_line_of_request
        for header in headers:
            request += '%s: %s\r\n' % (header, headers[header])
        request += '\r\n'

        print "request:\n", request
        response = self.get_remote_response(remote_host, remote_port, request)
        print "response:\n", response
        if response:
            data = None
            try:
                my_json = response
                data = json.loads(my_json.decode('utf-8'))
            except ValueError:
                message = 'Cannot parse JSON.'
                # self.send_error(400, message=message)
            filters = []
            for one_filter in data:
                filters.append((one_filter.get('filter_id'), one_filter.get('filter_name')))
        return filters

The binding, listening and accepting on the PROXY is done, so I think that I don't need to upload it here. If so, just say so :)

I've leaved the prints so you can see what is going to the database app:

The request:

request: GET http://127.0.0.1:9000/api/send-to-world/2/ HTTP/1.1 Host: 127.0.0.1 Content-Type: application/json Accept: application/json

And the response:

Trying to connect to:127.0.0.1 : 9000 response: Traceback (most recent call last): File "/home/dave/.virtualenvs/teleV1/local/lib/python2.7/site-packages/tornado/web.py", line 1446, in _execute result = self.prepare() File "/home/dave/.virtualenvs/teleV1/local/lib/python2.7/site-packages/tornado/web.py", line 2174, in prepare raise HTTPError(self._status_code) HTTPError: HTTP 404: Not Found

The logs from the database app while trying to get the filters:

tornado working on port 9000 WARNING:tornado.access:404 GET http://127.0.0.1:9000/api/send-to-world/2/ (127.0.0.1) 2.75ms

While I access it via the web browser it gives me the JSON.

So to sum up: How can I receive a JSON from my database app using a socket connection? I think that if the servers work on diffrent ports then the problem shouldn't be with the host: 127.0.0.1. If I'm wrong and I can't do some thing like this on one computer then please correct me :)

Upvotes: 0

Views: 199

Answers (1)

Solar.gy
Solar.gy

Reputation: 1157

I doubt this is the problem, but you said your database app listens on port 900, yet your proxy is connecting to port 9000...

Don't include the protocol, hostname and port in the HTTP request headers, so change

request_url = 'http://%s:%s/api/send-to-world/%s/' % (remote_host, str(remote_port), filter_id)

to

request_url = '/api/send-to-world/%s/' %filter_id

For example to make your proxy fetch this article from stackoverflow.com:

remote_host = 'stackoverflow.com'
remote_port = 80
request_url = 'posts/43481916'
first_line_of_request = 'GET ' + request_url + ' HTTP/1.1\r\n'
headers = {
    'Host': remote_host,
    'Content-Type': 'text/html',
    'Accept': 'text/html'
}
request = first_line_of_request
for header in headers:
    request += '%s: %s\r\n' % (header, headers[header])
request += '\r\n'
response = self.get_remote_response(remote_host, remote_port, request)

Upvotes: 1

Related Questions