CODEWITHSUNDEEP
pythontensorflow
Ghilas BELHADJ
Ghilas BELHADJ

Reputation: 14124

Disable Tensorflow debugging information

By debugging information I mean what TensorFlow shows in my terminal about loaded libraries and found devices etc. not Python errors.

I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcublas.so locally
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcudnn.so locally
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcufft.so locally
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcuda.so.1 locally
I tensorflow/stream_executor/dso_loader.cc:105] successfully opened CUDA library libcurand.so locally
I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:900] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
I tensorflow/core/common_runtime/gpu/gpu_init.cc:102] Found device 0 with properties: 
name: Graphics Device
major: 5 minor: 2 memoryClockRate (GHz) 1.0885
pciBusID 0000:04:00.0
Total memory: 12.00GiB
Free memory: 11.83GiB
I tensorflow/core/common_runtime/gpu/gpu_init.cc:126] DMA: 0 
I tensorflow/core/common_runtime/gpu/gpu_init.cc:136] 0:   Y 
I tensorflow/core/common_runtime/gpu/gpu_device.cc:717] Creating TensorFlow device (/gpu:0) -> (device: 0, name: Graphics Device, pci bus id: 0000:04:00.0)
I tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:51] Creating bin of max chunk size 1.0KiB
...

Upvotes: 359

Views: 348462

Answers (21)

Rudra shah
Rudra shah

Reputation: 841

For Apple M1 or Above GPU, you need to install package silence_tensorflow.

Run following command to install

python -m pip install silence-tensorflow

then add following lines in your code

from silence_tensorflow import silence_tensorflow
silence_tensorflow()

Upvotes: 1

stanly
stanly

Reputation: 103

This one worked for me perfectly, to turn off all the loggs

import logging, os

logging.disable(logging.WARNING)
os.environ["TF_CPP_MIN_LOG_LEVEL"] = "3"

import tensorflow as tf

Upvotes: 3

Teddy C
Teddy C

Reputation: 1016

Just run the silence_tensorflow function from silence-tensorflow package before importing tensorflow:

"""Module providing tools to shut up tensorflow useless warnings, letting you focus on the actual problems."""
import os
import logging


def silence_tensorflow():
    """Silence every unnecessary warning from tensorflow."""
    logging.getLogger('tensorflow').setLevel(logging.ERROR)
    os.environ["KMP_AFFINITY"] = "noverbose"
    os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
    # We wrap this inside a try-except block
    # because we do not want to be the one package
    # that crashes when TensorFlow is not installed
    # when we are the only package that requires it
    # in a given Jupyter Notebook, such as when the
    # package import is simply copy-pasted.
    try:
        import tensorflow as tf
        tf.get_logger().setLevel('ERROR')
        tf.autograph.set_verbosity(3)
    except ModuleNotFoundError:
        pass

Disclaimer: I'm not the author of this package.

Upvotes: 4

Matteo
Matteo

Reputation: 175

In Jupyter notebooks, you can use the %env magic command:

%env TF_CPP_MIN_LOG_LEVEL=3
import tensorflow as tf

Upvotes: 3

craymichael
craymichael

Reputation: 4821

2.0 Update (10/8/19) Setting TF_CPP_MIN_LOG_LEVEL should still work (see below in v0.12+ update), but there was a reported issue for version 2.0 until 2.3.z fixed in 2.4 and later. If setting TF_CPP_MIN_LOG_LEVEL does not work for you (again, see below), try doing the following to set the log level:

import tensorflow as tf
tf.get_logger().setLevel('INFO')

In addition, please see the documentation on tf.autograph.set_verbosity which sets the verbosity of autograph log messages - for example:

# Can also be set using the AUTOGRAPH_VERBOSITY environment variable
tf.autograph.set_verbosity(1)

v0.12+ Update (5/20/17), Working through TF 2.0+:

In TensorFlow 0.12+, per this issue, you can now control logging via the environmental variable called TF_CPP_MIN_LOG_LEVEL; it defaults to 0 (all logs shown) but can be set to one of the following values under the Level column.

  Level | Level for Humans | Level Description                  
 -------|------------------|------------------------------------ 
  0     | INFO             | [Default] Print all messages       
  1     | WARNING          | Filter out INFO messages           
  2     | ERROR            | Filter out INFO & WARNING messages 
  3     | NONE             | Filter out all messages

See the following generic OS example using Python:

import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'  # or any {'0', '1', '2'}
import tensorflow as tf

You can set this environmental variable in the environment that you run your script in. For example, with bash this can be in the file ~/.bashrc, /etc/environment, /etc/profile, or in the actual shell as:

TF_CPP_MIN_LOG_LEVEL=2 python my_tf_script.py

To be thorough, you call also set the level for the Python tf_logging module, which is used in e.g. summary ops, tensorboard, various estimators, etc.

# append to lines above
tf.logging.set_verbosity(tf.logging.ERROR)  # or any {DEBUG, INFO, WARN, ERROR, FATAL}

For 1.14 you will receive warnings if you do not change to use the v1 API as follows:

# append to lines above
tf.compat.v1.logging.set_verbosity(tf.compat.v1.logging.ERROR)  # or any {DEBUG, INFO, WARN, ERROR, FATAL}
**For Prior Versions of TensorFlow or TF-Learn Logging (v0.11.x or lower):**

View the page below for information on TensorFlow logging; with the new update, you're able to set the logging verbosity to either DEBUG, INFO, WARN, ERROR, or FATAL. For example:

tf.logging.set_verbosity(tf.logging.ERROR)

The page additionally goes over monitors which can be used with TF-Learn models. Here is the page.

This doesn't block all logging, though (only TF-Learn). I have two solutions; one is a 'technically correct' solution (Linux) and the other involves rebuilding TensorFlow.

script -c 'python [FILENAME].py' | grep -v 'I tensorflow/'

For the other, please see this answer which involves modifying source and rebuilding TensorFlow.

Upvotes: 251

Shahrokh Bah
Shahrokh Bah

Reputation: 410

After testing various suggestions so that they could also silence the resulting executable built with PyInstaller, I came up with this setting:

import logging
import os

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'
logging.getLogger('tensorflow').setLevel(logging.ERROR)

import tensorflow as tf

The line

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'

will silent the warning about rebuilding TensorFlow:

I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA. To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.

The line

logging.getLogger('tensorflow').setLevel(logging.ERROR)

will silent the warning about AutoGraph:

WARNING:tensorflow:AutoGraph is not available in this environment: functions lack code information. This is typical of some environments like the interactive Python shell. See https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/autograph/g3doc/reference/limitations.md#access-to-source-code for more information.

The key point is to place these two before importing Tensorflow—despite Pylint's warning!

tensorflow 2.11.0

Upvotes: 2

Bex T.
Bex T.

Reputation: 1796

Most of the answers here work, but you have to use them every time you open a new session (e.g. with JupyterLab). To make the changes stick, you have to set the environment variable.

Linux:

export TF_CPP_MIN_LOG_LEVEL="3"

(Also add the above line to .bashrc to make the change permanent, not just for the session)

Windows:

setx TF_CPP_MIN_LOG_LEVEL "3"

Both set the environment variables for the user.

Upvotes: 1

Ruli
Ruli

Reputation: 2791

I was struggling from this for a while, tried almost all the solutions here but could not get rid of debugging info in TF 1.14, I have tried following multiple solutions:

import os
import logging
import sys

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'  # FATAL
stderr = sys.stderr
sys.stderr = open(os.devnull, 'w')

import tensorflow as tf
tf.get_logger().setLevel(tf.compat.v1.logging.FATAL)
tf.compat.v1.logging.set_verbosity(tf.compat.v1.logging.ERROR)
logging.getLogger('tensorflow').setLevel(tf.compat.v1.logging.FATAL)

sys.stderr = stderr

import absl.logging
logging.root.removeHandler(absl.logging._absl_handler)
absl.logging._warn_preinit_stderr = False

The debugging info still showed up, what finally helped was restarting my pc (actually restarting the kernel should work). So if somebody has similar problem, try restart kernel after you set your environment vars, simple but might not come in mind.

Upvotes: 1

oreitor
oreitor

Reputation: 221

I am using Tensorflow version 2.3.1 and none of the solutions above have been fully effective.
Until, I find this package.

Install like this:

with Anaconda,

python -m pip install silence-tensorflow

with IDEs,

pip install silence-tensorflow

And add to the first line of code:

from silence_tensorflow import silence_tensorflow
silence_tensorflow()

That's It!

Upvotes: 22

Matt_Haythornthwaite
Matt_Haythornthwaite

Reputation: 753

To anyone still struggling to get the os.environ solution to work as I was, check that this is placed before you import tensorflow in your script, just like mwweb's answer:

import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'  # or any {'0', '1', '2'}
import tensorflow as tf

Upvotes: 17

Erman
Erman

Reputation: 1603

None of the solutions above could solve my problem in Jupyter Notebook, so I use the following snippet code bellow from Cicoria, and issues solved.

import warnings  
with warnings.catch_warnings():  
    warnings.filterwarnings("ignore",category=FutureWarning)
    import tensorflow as tf
    from tensorflow import keras
    from tensorflow.keras.preprocessing.text import Tokenizer

print('Done')

Upvotes: 1

masouduut94
masouduut94

Reputation: 1122

If you only need to get rid of warning outputs on the screen, you might want to clear the console screen right after importing the tensorflow by using this simple command (Its more effective than disabling all debugging logs in my experience):

In windows:

import os
os.system('cls')

In Linux or Mac:

import os
os.system('clear')

Upvotes: 0

Baycosinus
Baycosinus

Reputation: 106

for tensorflow 2.1.0, following code works fine.

import tensorflow as tf
tf.compat.v1.logging.set_verbosity(tf.compat.v1.logging.ERROR)

Upvotes: 3

serv-inc
serv-inc

Reputation: 38267

For compatibility with Tensorflow 2.0, you can use tf.get_logger 

import logging
tf.get_logger().setLevel(logging.ERROR)

Upvotes: 62

dturvene
dturvene

Reputation: 2406

Yeah, I'm using tf 2.0-beta and want to enable/disable the default logging. The environment variable and methods in tf1.X don't seem to exist anymore.

I stepped around in PDB and found this to work:

# close the TF2 logger
tf2logger = tf.get_logger()
tf2logger.error('Close TF2 logger handlers')
tf2logger.root.removeHandler(tf2logger.root.handlers[0])

I then add my own logger API (in this case file-based)

logtf = logging.getLogger('DST')
logtf.setLevel(logging.DEBUG)

# file handler
logfile='/tmp/tf_s.log'
fh = logging.FileHandler(logfile)
fh.setFormatter( logging.Formatter('fh %(asctime)s %(name)s %(filename)s:%(lineno)d :%(message)s') )
logtf.addHandler(fh)
logtf.info('writing to %s', logfile)

Upvotes: 2

Mandy007
Mandy007

Reputation: 441

I solved with this post Cannot remove all warnings #27045 , and the solution was:

import logging
logging.getLogger('tensorflow').disabled = True

Upvotes: 13

mwweb
mwweb

Reputation: 7925

You can disable all debugging logs using os.environ :

import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3' 
import tensorflow as tf

Tested on tf 0.12 and 1.0

In details, 

0 = all messages are logged (default behavior)
1 = INFO messages are not printed
2 = INFO and WARNING messages are not printed
3 = INFO, WARNING, and ERROR messages are not printed

Upvotes: 443

estevo
estevo

Reputation: 1011

Usual python3 log manager works for me with tensorflow==1.11.0:

import logging
logging.getLogger('tensorflow').setLevel(logging.INFO)

Upvotes: 6

Tyler
Tyler

Reputation: 28874

To add some flexibility here, you can achieve more fine-grained control over the level of logging by writing a function that filters out messages however you like:

logging.getLogger('tensorflow').addFilter(my_filter_func)

where my_filter_func accepts a LogRecord object as input [LogRecord docs] and returns zero if you want the message thrown out; nonzero otherwise.

Here's an example filter that only keeps every nth info message (Python 3 due to the use of nonlocal here):

def keep_every_nth_info(n):
    i = -1
    def filter_record(record):
        nonlocal i
        i += 1
        return int(record.levelname != 'INFO' or i % n == 0)
    return filter_record

# Example usage for TensorFlow:
logging.getLogger('tensorflow').addFilter(keep_every_nth_info(5))

All of the above has assumed that TensorFlow has set up its logging state already. You can ensure this without side effects by calling tf.logging.get_verbosity() before adding a filter.

Upvotes: 2

Wikunia
Wikunia

Reputation: 1592

As TF_CPP_MIN_LOG_LEVEL didn't work for me you can try:

tf.logging.set_verbosity(tf.logging.WARN)

Worked for me in tensorflow v1.6.0

Upvotes: 11

Pedro Lopes
Pedro Lopes

Reputation: 2883

I have had this problem as well (on tensorflow-0.10.0rc0), but could not fix the excessive nose tests logging problem via the suggested answers.

I managed to solve this by probing directly into the tensorflow logger. Not the most correct of fixes, but works great and only pollutes the test files which directly or indirectly import tensorflow:

# Place this before directly or indirectly importing tensorflow
import logging
logging.getLogger("tensorflow").setLevel(logging.WARNING)

Upvotes: 22

Related Questions