CODEWITHSUNDEEP
pythoncanalyzerpython-can
Wojciech Wołujewicz
Wojciech Wołujewicz

Reputation: 9

How to properly read J1939 messages from .asc file with cantools?

I'm trying to create a CAN logs converter from .asc files to .csv files (in human readable form). I'm somewhat successful. My code works fine with almost any database but j1939.dbc.

The thing is, that if I print out the messages read from the dbc file, I can see that the messages from j1939.dbc are read into the database. But it fails to find any of those messages in the processed log file. At the same time I can read the same file using Vector CANalyzer with no issues.

I wonder why this may happed and why it only affects the j1939.dbc and not the others.

I suspect that maybe the way I convert those messages is wrong because it never goes by the if msg_id in database: line (and as mentioned above, those messages are certainly there because Vector CANalyzer works fine with them).

EDIT: I realized that maybe the problem is not cantools but python-can package, maybe the can.ASCReader() doeasn't do well with j1939 frames and omits them? I'm gonna investigate myself but I hope someone better at coding will help.

import pandas as pd
import can
import cantools
import time as t
from tqdm import tqdm
import re
import os
from binascii import unhexlify


dbcs = [filename.split('.')[0] for filename in os.listdir('./dbc/') if filename.endswith('.dbc')]
files = [filename.split('.')[0] for filename in os.listdir('./asc/') if filename.endswith('.asc')]
start = t.time() 

db = cantools.database.Database()

for dbc in dbcs:
    with open(f'./dbc/{dbc}.dbc', 'r') as f:
        db.add_dbc(f)


f_num = 1

for fname in files:
    print(f'[{f_num}/{len(files)}] Parsing data from file: {fname}')    
    log=can.ASCReader(f'./asc/{fname}.asc')
    entries = []
    all_msgs =[]


    message = {'Time [s]': ''}
    database = list(db._frame_id_to_message.keys())
    print(database)
    lines = sum(1 for line in open(f'./asc/{fname}.asc'))
    msgs = iter(log)

    try:
        for msg, i in zip(msgs, tqdm(range(lines))):

            msg = re.split("\\s+", str(msg))
            timestamp = round(float(msg[1]), 0)
            msg_id = int(msg[3], 16)

            try:
                data = unhexlify(''.join(msg[7:15]))
            except:
                continue

            if msg_id in database:
                if timestamp != message['Time [s]']:
                    entries.append(message.copy())
                    message.update({'Time [s]': timestamp})
                message.update(db.decode_message(msg_id, data))

    except ValueError:
        print('ValueError')
        
    df = pd.DataFrame(entries[1:])
    duration = t.time() - start
    df.to_csv(f'./csv/{fname}.csv', index=False)
    print(f'DONE IN {int(round(duration, 2)//60)}min{round(duration % 60, 2)}s!\n{len(df.columns)} signals extracted!')
    f_num += 1

Upvotes: 0

Views: 2273

Answers (1)

richard
richard

Reputation: 11

class can.ASCReader(file, base=’hex’) Bases: can.io.generic.BaseIOHandler Iterator of CAN messages from a ASC logging file. Meta data (comments, bus statistics, J1939 Transport Protocol messages) is ignored. Might answer your question...

Upvotes: 1

Related Questions