Rancher - How to expose my services publicly?

Question

I have a running Rancher setup as the following:

• Host machine (running rancher/rancher container) on a public IP;
• Nodes in an exclusive network (10.1.1.0/24) not accessible from the Internet.

My goal is to serve a web application using the Rancher Load Balancing or whatever similar stuff. For that, I've perfomed the following steps:

• Deploy workload using the "rancher/hello-world" image on 3 pods mapping port 80/tcp as a NodePort (listening port is random) named "web-app";
• Add Ingress named "hello" in same namespace selecting "Automatically generate a .xip.io hostname" and adding route "/" to the "web-app" on port 80.

This works on the local network, since I get an address like http://hello.gabriel-milan.10.1.1.14.xip.io/ that would resolve to 10.1.1.14, which is local.

I wanted to expose this service for one of my public IPs. How can I do that?

Answer 1

Edit 2021-09-27: xip.io is gone, but I'm leaving those references in my response because the OP asked about xip.io. Alternatives are sslip.io and nip.io, which both function the same way. You can replace xip.io in my response with either of those to achieve the same results.

There are a couple ways of doing this. Based on your use of a private network that is not accessible from the Internet, the nodes don't have public IPs, and therefore Kubernetes doesn't know anything about whatever public IP is mapped to them. This is how it works in EC2, or anywhere that has a NAT happening off the nodes.

If those nodes are a Custom cluster (where you install Docker and then use the docker run command from Rancher to install RKE and join the cluster to Rancher), then before you install, you can click the Advanced Options link in the bottom right corner and set the public and private IPs for each node.

When you do this, the nodes receive a label that holds the public IP, and that address will be used with your xip.io hostname that you generate when setting up the Ingress.

Without that label, the xip.io hostname picks up the primary IP of the node, which in this case is on the private network.

If you do this, though, your traffic will only go to one node on the cluster, even if your ingress controller is listening on multiple nodes.

Instead, when running a multi-node cluster, I recommend that you put a Layer 4 load balancer in front of all worker nodes (or the nodes where the ingress controller is listening if it's not listening on every node). Punch through 80 and 443, and then use that as the target for your domain.

domain.com -> load balancer -> ingress controller (on all nodes) -> Service -> Pods

Your ingress controller is listening on 80/443 for HTTP traffic, which also means that your Service doesn't have to be NodePort. It can be ClusterIP because traffic goes through the ingress controller and then is routed inside the cluster.

NodePort services are used when you have an external load balancer and you need to direct traffic to a specific service. In that scenario, the external load balancer replaces the ingress controller. You create NodePort services for each of your apps, and then you tell the load balancer to send traffic for App A to each node on port 30547 or whatever the NodePort is for that service.

Incidentally, if you're in a cloud provider, you can combine these into a LoadBalancer Service. That will create a NodePort Service on the nodes and then reach out to the cloud provider's API and deploy a cloud load balancer and then program it with the nodes, the port for the Service, and maintain that configuration for the life of the Service.

To recap:

• Your nodes don't know their public IP, so the xip.io hostname can't know it either
• Put a Layer 4 load balancer in front of your nodes and send traffic to 80/443 on all nodes
• Change your Service to be ClusterIP
• Send traffic to the load balancer

Also, as a workaround if you don't want to deploy a load balancer, you can delete the Ingress and recreate it, but instead of creating an xip.io hostname automatically, choose "Set a hostname" and create it manually. If one node's public IP is 1.2.3.4, then you can set it to any.thing.you.want.1.2.3.4.xip.io and it'll return 1.2.3.4 to DNS queries.

You just can't edit an existing xip.io Ingress and change it to a different manual xip.io hostname. You have to recreate it.

In this workaround traffic is still coming in to the ingress controller, so you can still change your Service from NodePort to ClusterIP.

Disclosure: I work for Rancher.

Answer 2

With everything on bare metal-metal in a traditional home network using a router, I wanted to expose apps on the web with only limited points of entry into the cluster (ports 80 and 443 for http and https traffic respectively), without opening any more ports than necessary and keeping workloads and services safe behind ClusterIP networking (no need to expose ports for workloads and services with NodePort).

None of my nodes have a public IP, only my router does.

Ingress was the answer, as it can route traffic based on different hostnames all the while using the same IP address.

Because the nodes in my cluster were all inside a private home network, there were no straightforward instructions on how to do this.

Additionally, I wanted to use xip.io to generate free URLs for testing purposes that worked over the WWW, but the Automatically generate a .xip.io hostname option when adding ingress through the Rancher UI would only work on my local network, not the web.

I banged my head on this for days.

But I finally figured it out and it's easy as pie.

First, we need to go into the router's settings and do some port forwarding.

Say your router's public IP is 16.43.236.65 and you have a node on your cluster whose internal private IP on the home network is 192.168.0.155

In your router's settings, forward all port 80 and 443 traffic to ports 80 and 443 of one of your nodes so that it essentially looks like something like this:

16.43.236.65:80 -> 192.168.0.155:80

16.43.236.65:443 -> 192.168.0.155:443

Once that is done, go back to Rancher and click on Add Ingress under Load Balancing, unfortunately Automatically generate a .xip.io hostname will still use your node's private IP in your home network and only work locally.

Instead, you can just select Specify a hostname to use and use your public IP with xip.io like so:

16.43.236.65.xip.io

Now configure the rest of your ingress, I just leave everything at their defaults for now, I use Service instead of Workload for Target Backend.

Now, when navigating to 16.43.236.65.xip.io on your browser, you should be able to access the service you selected in the Ingress you added over the web.

If you want to add other services using xip.io you can, just add subdomains, for example:

another-service.16.43.236.65.xip.io

yet-another-service.16.43.236.65.xip.io

It might look like magic but the way it works is by hostname, so you can literally have any domain name point to the same public IP, Ingress will know where to forward the traffic to and which service/workload to use based on the hostname, which I find brilliant. It might look like magic but the way it works is by hostname, so you can literally have any domain name point to the same public IP, Ingress will know where to forward the traffic and which service/workload to use based on the hostname, which I find brilliant.

Lastly, if you want to use an actual domain name: in the DNS settings from your domain name provider set an A Record that points to your Router's public IP. Using our current example that would be 16.43.236.65.

On Rancher add an Ingress and specify that same domain name.

Same concept. Hopes this helps!

Answer 3

1. You have to create a svc with type:LoadBalancer. It will give a public IP for your deployment. kubectl create service loadbalancer <deployment-name> --tcp=80:8000
2. once you create svc, you have to create ingress and pass above created svc in ingress. You have to define host: as a url of the domain and path where you want to map it.
3. Point the LB's IP in DNS provider.