Plotly Express only displays part of lines on graph, but data still shows when hovering

Question

I am using px.line to create a line plot and I also add another trace via go.Scatter(). In this case I have multiple plots in the same HTML file. For some of my graphs, only some of the data displays. However, when I hover, the actually datapoints of all the traces clearly displays (though you can't see the line itself).

Details on my code:

# some variables are defined elsewhere (EX: title, ylabel), but here are all the actions I'm performing on the figure

# CREATE figure
fig = px.line(data, title=title, render_mode='svg', markers=False)
# UPDATE layout
fig.update_layout(yaxis_title=ylabel, xaxis_title=xlabel)
# FORMAT hover data
fig.update_traces(xhoverformat=xhoverformat, yhoverformat=yhoverformat, hovertemplate=hovertemplate)
# Add last trace
fig.add_trace(go.Scatter(x=df.index, y=df['my_column'], name='my name', mode='lines', line=dict(dash='longdash', color='black')))
# FORMAT hover data - again to accommodate last trace that is only applied sometimes
fig.update_traces(xhoverformat=xhoverformat, yhoverformat=yhoverformat, hovertemplate=hovertemplate)
fig.update_layout(yaxis=dict(autorange=True, fixedrange=False), xaxis=dict(rangeslider=dict(visible=True), range=range))
# I add multiple annotations and vlines
# fig.add_vline(x=my_val)
# fig.add_annotation()

# it is then added to a list of figures and repeated
fig_list.append(fig)

# I then use the following function to write the figure to file
def plotly_figs_to_html(fig_list,filepath, plotlyjs = 'cdn'):
    """
    Give a list of plotly figures, saves the figures to an html file.

    :param fig_list: List (of plotly figures)
    :param filepath: String (of output filepath - expected to end in .html)
    :param plotlyjs: String (of optional plotlyjs type)
    :return: None
    """
    if len(fig_list) > 0:
        #Open the file
        with open(filepath, 'w') as f:
            #write first figure w/ full html and plotlyjs:
            f.write(fig_list[0].to_html(full_html = True, include_plotlyjs = plotlyjs))
            for fig in fig_list[1:]:
                #loop over rest of figures and add each w/o full html or js
                f.write(fig.to_html(full_html = False, include_plotlyjs = False))
    else:
        logging.warning(f'  No data to plot; {filepath.rsplit("/",1)[-1]}')
    return None

plotly_figs_to_html(fig_list, _filepath)

I'm guessing this is related to one or a combination of the following factors:

• Number of traces on the plot
• Number of datapoints on a plot
• Number of plots in the file or Number of datapoints in the file

Some notes:

• This file regularly has > 8 files, so I am purposefully using SVG over webGL
• The first figure has full_html=True and include_plotlyjs='cdn', and all subsequent figures have both parameters set to False.
• The number of days and traces can vary greatly. So while normally this can be easily combined into one file of < 20 MB, I am seeing this issue on a file that is 87MB
• The dataframes going into px.line have sizes ranging from 40k to 182k
  • While the interactivity slows down w/ svg and this many points, it's worth it for me to group the plots
  • This file has ~25 plots with the average size of the dataframse being 110k
• Datapoints are Float68
• datetime index is sorted
• there are often NaNs for each trace
• The go.Scatter() trace shows even when the traces created with px.line do not

Does anyone know what is the cause of this? I'm going to work on mitigating the factors above, but it would be really helpful to know which (if any) is the cause.

pandas 1.4.4 plotly 5.9.0 python 3.9.13

Update 1: reduce df.size by about 40% I just removed many Nans (I knew I had indices with consistently no data). This dropped the file size to 50MB and each df has about 65k points and the problem still occurs.

Update 2 As seen in my answer below, I can see the points by adding markers. The data not being shown is when the Series is in the form: [Nan, some_value, Nan]. However, rendering markers with my size of file and/or plots crashes the HTML file. Anyone know of another alternative?

Answer 1

Okay, so I think I figured out what is happening. It is common to have Nans throughout my data. And if a point on the trace is sandwiched by Nans on either side, then we don't see it. Reproducing this with a simple example:

import numpy as np
import pandas as pd
import plotly.express as px

#Create DF with some values having a Nan on either side
df = pd.DataFrame({'a':[np.nan,1, np.nan,1.1,1.2, np.nan,1.3],'b':[np.nan,2,2.2,2.2, np.nan,1, np.nan]})

fig=px.line(df)
fig.show()

In the resulting graph we see two short lines, representing 5 of the total datapoints:

Repeating the same code, but with fig=px.line(df, markers=True), we see the additional points that are sandwhiched by Nans:

So far, this leaves me with a few options:

1. Add Markers.
   Two concerns: 1) this isn't the look I want. 2) I'm not sure if rendering the markers will inflate the files size or slow the interactivity of my (data-heavy) plots.
2. Loop over the columns, dropNA indices, then add each trace individually The major problem here is that it will connect all the points (where I would rather see my gaps in data represented as gaps in the traces).

UPDATE While this will display the data, it is not a good solution for the amount of data I am displaying. It is taking an extremely long time for the file to load. The file is the same size as the 'lines' plot, but loading the plots and interacting with the data is pretty much impossible.

I eventually get a Chrome error: "Error code: Out of Memory"