CODEWITHSUNDEEP
pythonsyntax-highlightingsublimetextpython-loggingsublime-syntax
Serhat
Serhat

Reputation: 11

Sublime Syntax Highlighting for Custom Python Logging Formatter

I want to highlight my python log outputs with sublime syntax definitions according to my custom python logging formatter. I try below syntax definition but it did not work. I can't see this highlight on my sublime highlight list. Below is my python CustomFormatter class, sublime syntax definitions and example log. Can you give me any solution?

My syntax definition is:

%YAML 1.2
---
# See http://www.sublimetext.com/docs/syntax.html
name: TBLog
file_extensions:
  - tblog
scope: source.tblog
contexts:
  main:
    - match: '^\[.*\].*'
      captures:
        '0': { name: string.timestamp.tblog }
        '1': { name: string.loglevel.tblog }
      push: log_message

  log_message:
    - match: '$'
      pop: true
    - include: string.quoted.double
    - include: string.quoted.single
    - include: keyword.control
    - include: constant.numeric
    - include: string.timestamp

  string.timestamp:
    - match: '\[\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}\]'
      scope: string.timestamp.tblog

  string.quoted.double:
    - match: '"'
      scope: punctuation.definition.string.begin.tblog
      push:
        - meta_scope: string.quoted.double.tblog
        - match: '\\.'
          scope: constant.character.escape.tblog
        - match: '"'
          scope: punctuation.definition.string.end.tblog
          pop: true

  string.quoted.single:
    - match: "'"
      scope: punctuation.definition.string.begin.tblog
      push:
        - meta_scope: string.quoted.single.tblog
        - match: '\\.'
          scope: constant.character.escape.tblog
        - match: "'"
          scope: punctuation.definition.string.end.tblog
          pop: true

  keyword.control:
    - match: '\b(if|else|for|while)\b'
      scope: keyword.control.tblog

  constant.numeric:
    - match: '\b(-)?[0-9.]+\b'
      scope: constant.numeric.tblog

  string.loglevel:
    - match: '\[(WARNING|ERROR|CRITICAL|DEBUG|INFO)\]'
      captures:
        '0': { name: string.loglevel.tblog }
      scope: meta.loglevel.tblog
      push: log_message

  meta_scope:
    string.loglevel.tblog: 'meta.loglevel.tblog'

My custom python formatter is:

class ColorFormatter(logging.Formatter):
    COLORS = {
        'WARNING': colorama.Fore.YELLOW + colorama.Style.BRIGHT,
        'ERROR': colorama.Fore.RED + colorama.Style.BRIGHT,
        'CRITICAL': colorama.Fore.WHITE + colorama.Style.BRIGHT + colorama.Back.RED,
        'DEBUG': colorama.Fore.BLUE + colorama.Style.BRIGHT,
        'INFO': colorama.Fore.GREEN + colorama.Style.BRIGHT,
        'DEFAULT': colorama.Fore.WHITE,
    }

    def __init__(self, fmt="[%(asctime)s][%(levelname).1s]%(message)s", datefmt="%Y-%m-%d %H:%M:%S"):
        logging.Formatter.__init__(self, fmt, datefmt)

    def format(self, record):
        color = self.COLORS.get(record.levelname, self.COLORS['DEFAULT'])
        message = logging.Formatter.format(self, record)
        return f"{color}{message}\033[0m"

Example log is:

[2023-02-23 22:58:25,226][I]TC] Initializing the bot!
[2023-02-23 22:58:25,241][D]EX] Creating exchange client.
[2023-02-23 22:58:25,678][D]EX] Exchange client has been created!
[2023-02-23 22:58:25,709][I]TC] Preparing bot to start trading!
[2023-02-23 22:58:25,725][D]TC] Variables has been reset!
[2023-02-23 22:58:26,023][D]TC] Server time has been updated! [ST (ms): 1677182307137]
[2023-02-23 22:58:26,038][D]TC] Time difference has been updated! [TD (ms): 1114]
[2023-02-23 22:58:26,337][D]TC] Account order asset balance has been updated! [B: 64.94 BUSD]
[2023-02-23 22:58:26,353][D]TC] Buy order size has been updated! [BOS: 64 BUSD]
[2023-02-23 22:58:26,385][I]TC] Scanning for potential entry opportunities!

Upvotes: 1

Views: 397

Answers (2)

Serhat
Serhat

Reputation: 11

I used below YAML syntax code.

%YAML 1.2
---
# See http://www.sublimetext.com/docs/syntax.html
name: TBLog
file_extensions:
  - tblog
scope: tb
contexts:
  log_line:
      # data
    - match: '\[([^\[\]]+)\]'
      scope: tb.data
      push: log_line
              
    - match: '$'
      pop: true

  main:
    # debug
    - match: '\[(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2})\]\[D\]'
      scope: tb.debug
      push: log_line

    # info
    - match: '\[(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2})\]\[I\]'
      scope: tb.info
      push: log_line

    # warning
    - match: '\[(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2})\]\[W\]'
      scope: tb.warning
      push: log_line

    # error
    - match: '\[(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2})\]\[E\]'
      scope: tb.error
      push: log_line

    # critical
    - match: '\[(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2})\]\[C\]'
      scope: tb.critical
      push: log_line

    # datetime
    - match: '\[(\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2})\]'
      scope: tb.datetime
      push:
        - match: '\[D\]'
          scope: tb.debug
          pop: true
        - match: '\[I\]'
          scope: tb.info
          pop: true
        - match: '\[W\]'
          scope: tb.warning
          pop: true
        - match: '\[E\]'
          scope: tb.error
          pop: true
        - match: '\[C\]'
          scope: tb.critical
          pop: true

And below tmTheme.xml for colors.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>name</key>
    <string>TBLog</string>
    <key>settings</key>
    <array>

      <dict>
        <key>settings</key>
        <dict>
          <key>background</key>
          <string>#272822</string>
          <key>caret</key>
          <string>#F8F8F0</string>   
          <key>foreground</key>
          <string>#F8F8F2</string>
          <key>invisibles</key>
          <string>#5E5E5E</string>
          <key>line_highlight</key>
          <string>#3E3D32</string>
          <key>selection</key>
          <string>#49483E</string>
          <key>findHighlightForeground</key>
          <string>#000000</string>
          <key>selectionBorder</key>
          <string>#222218</string>
          <key>activeGuide</key>
          <string>#9D550FB0</string>
          <key>bracketsForeground</key>
          <string>#F8F8F2A5</string>
          <key>bracketsOptions</key>
          <string>underline</string>
          <key>bracketsContentsForeground</key>
          <string>#F8F8F2A5</string>
          <key>bracketsContentsOptions</key>
          <string>underline</string>
          <key>tagsOptions</key>
          <string>stippled_underline</string>
        </dict>
      </dict>

      <dict>
        <key>name</key>    
        <string>Datetime</string>
        <key>scope</key>    
        <string>tb.datetime</string>     
        <key>settings</key>    
        <dict>
          <key>foreground</key>
          <string>#00B764</string>
        </dict>
      </dict>

      <dict>
        <key>name</key>    
        <string>Data</string>
        <key>scope</key>    
        <string>tb.data</string>     
        <key>settings</key>    
        <dict>
          <key>foreground</key>
          <string>#FFA500</string>
        </dict>
      </dict>

      <dict>
        <key>name</key>    
        <string>Debug</string>
        <key>scope</key>    
        <string>tb.debug</string>     
        <key>settings</key>    
        <dict>
          <key>foreground</key>
          <string>#16ACBA</string>
        </dict>
      </dict>

      <dict>
        <key>name</key>    
        <string>Info</string>
        <key>scope</key>    
        <string>tb.info</string>     
        <key>settings</key>    
        <dict>
          <key>foreground</key>
          <string>#00B764</string>
        </dict>
      </dict>

      <dict>
        <key>name</key>    
        <string>Warning</string>
        <key>scope</key>    
        <string>tb.warning</string>     
        <key>settings</key>    
        <dict>
          <key>foreground</key>
          <string>#EDD436</string>
        </dict>
      </dict>

      <dict>
        <key>name</key>    
        <string>Error</string>
        <key>scope</key>    
        <string>tb.error</string>     
        <key>settings</key>    
        <dict>
          <key>background</key>
          <string>#FF0000</string>
          <key>foreground</key>
          <string>#FFFFFF</string>
        </dict>
      </dict>

      <dict>
        <key>name</key>    
        <string>Critical</string>
        <key>scope</key>    
        <string>tb.critical</string>     
        <key>settings</key>    
        <dict>
          <key>background</key>
          <string>#FF0000</string>
          <key>foreground</key>
          <string>#FFFFFF</string>
        </dict>
      </dict>

    </array>
</dict>
</plist>

And below is how its look like.

https://i.sstatic.net/T8iL7.jpg

Upvotes: 0

Keith Hall
Keith Hall

Reputation: 16095

If you open the Sublime Text console (View menu -> Show Console), you will see an error when you save your sublime-syntax file:

error parsing lexer: Packages/User/tblog.sublime-syntax: patterns must be a vector at line 67 column 5

Line 67 is the line under your meta_scope context.

Removing that context, as it isn't referenced anywhere, gives us:

error parsing lexer: Packages/User/tblog.sublime-syntax: capture scope must be a string at line 10 column 7

If you look at the example captures in the documentation, you'll see how you shouldn't string quote the capture group numbers and it expects a plain string instead of a map with a name:.

After fixing all that, it still won't highlight the way you might expect because of the .* you have at the end of your first match pattern in the main context. Removing that gives:

%YAML 1.2
---
# See http://www.sublimetext.com/docs/syntax.html
name: TBLog
file_extensions:
  - tblog
scope: source.tblog
contexts:
  main:
    - match: '^\[.*\]'
      captures:
        0: string.timestamp.tblog
        1: string.loglevel.tblog
      push: log_message

  log_message:
    - match: '$'
      pop: true
    - include: string.quoted.double
    - include: string.quoted.single
    - include: keyword.control
    - include: constant.numeric
    - include: string.timestamp

  string.timestamp:
    - match: '\[\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}\]'
      scope: string.timestamp.tblog

  string.quoted.double:
    - match: '"'
      scope: punctuation.definition.string.begin.tblog
      push:
        - meta_scope: string.quoted.double.tblog
        - match: '\\.'
          scope: constant.character.escape.tblog
        - match: '"'
          scope: punctuation.definition.string.end.tblog
          pop: true

  string.quoted.single:
    - match: "'"
      scope: punctuation.definition.string.begin.tblog
      push:
        - meta_scope: string.quoted.single.tblog
        - match: '\\.'
          scope: constant.character.escape.tblog
        - match: "'"
          scope: punctuation.definition.string.end.tblog
          pop: true

  keyword.control:
    - match: '\b(if|else|for|while)\b'
      scope: keyword.control.tblog

  constant.numeric:
    - match: '\b(-)?[0-9.]+\b'
      scope: constant.numeric.tblog

  string.loglevel:
    - match: '\[(WARNING|ERROR|CRITICAL|DEBUG|INFO)\]'
      captures:
        '0': string.loglevel.tblog
      scope: meta.loglevel.tblog
      push: log_message

and then your text looks like this in ST with your TBLog syntax enabled: example TBLog highlighting

Upvotes: 1

Related Questions