How to locate Google Cloud Platform VM Logs?

Question

I am trying to find the past logs for the IPs that are accessing my GCP VM (by date if possible). Where do I find this?

I am familiar with the logs explorer feature but am not able to locate the IPs that tried to access the VM but were blocked by the firewall/security .

My goal is to keep track of these IPs for some sort of blacklisting.

Do I need to set something up in order to be able to track these IPs?

Edit 1 : Logging and access control is relatively new to me(please be patient), here is what I found out so far.

1. Firewall rule logging is now enabled for the main rule that blocks external access.
2. Web server logging is enabled and I am currently a logging admin.
3. Some background : I was working on Jupyter Notebook on an instance and I noticed a suspicious IP making a weird GET call in the Jupyter log(with a 404 response). I instantly freaked out and tightened security as per my knowledge but I do want to get to the root of this.
4. For now here is what I found out about that access attempt.

4.a - Firewall 'Resource Type' in the logs explorer did not catch/record the weird access attempt in any way ( But it did log my successful access attempts for that day, weird)

4.b - Around the time I noticed the IP, there are two IntegrityEvents recorded in the VM Instance 'Resource Type'. ShieldedVMIntegrity? Could this have something to do with that? It did not tell me the IP address though. (I will try to understand Shielded VMs better in the meantime. )

4.c - It feels like luck that I was looking at the logs when the weird API call was made, will double down on a GCP security course.

Also, how did you guys learn about stuff like this?

Edit 2 :

Okay so now I am able to track IPs trying to access through that firewall rule. I am seeing a lot of international IPs from Rus, China etc. Is that normal?

Don't you need like a key to access the SSH port, how are these IPs able to get through my firewall?

Sorta had a mini freak out when I saw the traffic in the images. Please note that the traffic(image) was recorded when I did not have any IP filter for the SSH port. What do these IPs getting access mean? Are my keys compromised?

Can you please help me get to the resource where I can find how to create a firewall correctly? I tried setting the ingress to my IP only but that doesn't seem to be working.

I also tried replicating @Jose 's firewall rule by limiting access to get a "DENIED" but am not able to do that either. Sorry if I seem like a beginner.

Any help is appreciated.

Thank you!

Answer 1

Adding to @Jose's answer above, here are some helpful pointers

Here is how I was able to figure out logging and access control

• https://cloud.google.com/vpc/docs/firewalls

• As per my understanding, there are automated softwares(with IP rotation I think) that constantly try to access your VM . These are the IPs I was noticing on my logs.

  • Make sure to remove unnecessary firewall rules.
  • If you are curious about the types of IPs trying to access your VM, it might be a good exercise to create a DENY(NOT ALLOW) rule for all ports and all IPs that is higher than all other rules, then turn on logging and you will see these IPs on the logging console. Since all IPs other than the ones allowed are just auto kicked by GCP default firewalls, you dont see them normally. Please make sure to do your DD before doing this.
  • Please turn on VM features like SecureBoot and Integrity monitoring as they will act like a another layer of protection.

• For secure access to VM instance , please refer to these official sources

  • https://cloud.google.com/solutions/connecting-securely
  • https://cloud.google.com/iap/docs/using-tcp-forwarding#gcloud_3

Let me know if you have any questions.

Answer 2

If you have your Firewall Rules Logging enable you will be able to see the IPs that are accessing your GCP VM. If not you can enable your firewall rules logging as @jabbson mentioned.

When you enable logging for a firewall rule, Google Cloud creates an entry called a connection record each time the rule allows or denies traffic. You can view these records in Cloud Logging, and you can export logs to any destination that Cloud Logging export supports.

Each connection record contains the source and destination IP addresses, the protocol and ports, date and time, and a reference to the firewall rule that applied to the traffic.

Please take in consideration that it will create a lot of logs, and since it is defined at the firewall you can filter it by port in order to avoid receiving thousands of logs.

To enabling firewall rules logging you could use the following command:

gcloud compute firewall-rules update NAME \
    --enable-logging
    --logging-metadata=LOGGING_METADATA

Where NAME is the name of the firewall rule and LOGGING_METADATA specifies whether Firewall Rules Logging includes metadata fields in firewall rule logs.

Edit 1

After you activated your Firewall Rules Logging you can see the information collected in this way:

Go to the Firewall page in the Google Cloud Console. Then click on the name of the rule you want to see the logs, and look for the logs section and click on view in Logs Explorer:

Then you could adjust the query to your needs.

In my example I created a firewall rule for my port 22 (ssh) and then I filtered by jsonPayload.disposition="DENIED"

Then you will able to see the IPs under the connection area