ggplot smoothing optimisation for high resolution data

Question

I'm looking at various smoothing options for some noisy, high resolution data and I'm wondering if ggplot2 geom_smooth() can be optimised to give the results I'm after.

The default 'gam' method produces less that adequate results, and the 'loess' method can't handle the amount of data. Are there better ways to handle this?

p <- ggplot(data=fid_df,aes(x=rt,y=value, group=1,text=paste("height = ", value, " RT = ",round(rt,3)))) +
    geom_line() +
    geom_smooth(aes(colour='red')) +
    xlim(10,55) +
    ylim(8, ceiling(max(fid_df$value[fid_df$rt>10])/10)*10)

A better (but still not ideal) solution for smoothing was obtained using the base plot function and colMeans, which captures the shape of the original data and eliminates a little bit of the noise, but not enough:

x<-fid_df$rt
y<-fid_df$value
plot(x,y, xlim=c(8,max(x)),ylim=c(8,max(y[x>8])*1.1),type="l")
k <- 20 #the number of rows to average over for smoothing
sig.new <- colMeans(matrix(y, nrow=k))
x.new <- colMeans(matrix(x, nrow=k))
lines(x.new,sig.new, col='lightslateblue')

A close up of the baseline shows the noise reduction achieved by this method:

plot(x,y, xlim=c(18,23),ylim=c(8.8,8.95),type="l")
lines(x.new,sig.new, col='red')

The ideal result would maintain the shape of actual peaks while eliminating noise especially from the baseline. The end objective is to distinguish the actual peaks from the baseline within a given tolerance in order to measure the area of each peak. Something like this: (generated in different software):

A subset of the data (the full data has almost 72,000 rows):

> dput(fid_df[20200:20800,])
structure(list(rt = c(16.832558914251, 16.833392252667, 16.834225591083, 
16.835058929499, 16.8358922679151, 16.8367256063311, 16.8375589447471, 
16.8383922831631, 16.8392256215791, 16.8400589599952, 16.8408922984112, 
16.8417256368272, 16.8425589752432, 16.8433923136592, 16.8442256520752, 
16.8450589904913, 16.8458923289073, 16.8467256673233, 16.8475590057393, 
16.8483923441553, 16.8492256825713, 16.8500590209874, 16.8508923594034, 
16.8517256978194, 16.8525590362354, 16.8533923746514, 16.8542257130674, 
16.8550590514835, 16.8558923898995, 16.8567257283155, 16.8575590667315, 
16.8583924051475, 16.8592257435635, 16.8600590819796, 16.8608924203956, 
16.8617257588116, 16.8625590972276, 16.8633924356436, 16.8642257740596, 
16.8650591124757, 16.8658924508917, 16.8667257893077, 16.8675591277237, 
16.8683924661397, 16.8692258045557, 16.8700591429718, 16.8708924813878, 
16.8717258198038, 16.8725591582198, 16.8733924966358, 16.8742258350518, 
16.8750591734679, 16.8758925118839, 16.8767258502999, 16.8775591887159, 
16.8783925271319, 16.8792258655479, 16.880059203964, 16.88089254238, 
16.881725880796, 16.882559219212, 16.883392557628, 16.884225896044, 
16.8850592344601, 16.8858925728761, 16.8867259112921, 16.8875592497081, 
16.8883925881241, 16.8892259265401, 16.8900592649562, 16.8908926033722, 
16.8917259417882, 16.8925592802042, 16.8933926186202, 16.8942259570363, 
16.8950592954523, 16.8958926338683, 16.8967259722843, 16.8975593107003, 
16.8983926491163, 16.8992259875324, 16.9000593259484, 16.9008926643644, 
16.9017260027804, 16.9025593411964, 16.9033926796124, 16.9042260180285, 
16.9050593564445, 16.9058926948605, 16.9067260332765, 16.9075593716925, 
16.9083927101085, 16.9092260485246, 16.9100593869406, 16.9108927253566, 
16.9117260637726, 16.9125594021886, 16.9133927406046, 16.9142260790207, 
16.9150594174367, 16.9158927558527, 16.9167260942687, 16.9175594326847, 
16.9183927711007, 16.9192261095168, 16.9200594479328, 16.9208927863488, 
16.9217261247648, 16.9225594631808, 16.9233928015968, 16.9242261400129, 
16.9250594784289, 16.9258928168449, 16.9267261552609, 16.9275594936769, 
16.9283928320929, 16.929226170509, 16.930059508925, 16.930892847341, 
16.931726185757, 16.932559524173, 16.933392862589, 16.9342262010051, 
16.9350595394211, 16.9358928778371, 16.9367262162531, 16.9375595546691, 
16.9383928930851, 16.9392262315012, 16.9400595699172, 16.9408929083332, 
16.9417262467492, 16.9425595851652, 16.9433929235812, 16.9442262619973, 
16.9450596004133, 16.9458929388293, 16.9467262772453, 16.9475596156613, 
16.9483929540774, 16.9492262924934, 16.9500596309094, 16.9508929693254, 
16.9517263077414, 16.9525596461574, 16.9533929845735, 16.9542263229895, 
16.9550596614055, 16.9558929998215, 16.9567263382375, 16.9575596766535, 
16.9583930150696, 16.9592263534856, 16.9600596919016, 16.9608930303176, 
16.9617263687336, 16.9625597071496, 16.9633930455657, 16.9642263839817, 
16.9650597223977, 16.9658930608137, 16.9667263992297, 16.9675597376457, 
16.9683930760618, 16.9692264144778, 16.9700597528938, 16.9708930913098, 
16.9717264297258, 16.9725597681418, 16.9733931065579, 16.9742264449739, 
16.9750597833899, 16.9758931218059, 16.9767264602219, 16.9775597986379, 
16.978393137054, 16.97922647547, 16.980059813886, 16.980893152302, 
16.981726490718, 16.982559829134, 16.9833931675501, 16.9842265059661, 
16.9850598443821, 16.9858931827981, 16.9867265212141, 16.9875598596301, 
16.9883931980462, 16.9892265364622, 16.9900598748782, 16.9908932132942, 
16.9917265517102, 16.9925598901262, 16.9933932285423, 16.9942265669583, 
16.9950599053743, 16.9958932437903, 16.9967265822063, 16.9975599206224, 
16.9983932590384, 16.9992265974544, 17.0000599358704, 17.0008932742864, 
17.0017266127024, 17.0025599511185, 17.0033932895345, 17.0042266279505, 
17.0050599663665, 17.0058933047825, 17.0067266431985, 17.0075599816146, 
17.0083933200306, 17.0092266584466, 17.0100599968626, 17.0108933352786, 
17.0117266736946, 17.0125600121107, 17.0133933505267, 17.0142266889427, 
17.0150600273587, 17.0158933657747, 17.0167267041907, 17.0175600426068, 
17.0183933810228, 17.0192267194388, 17.0200600578548, 17.0208933962708, 
17.0217267346868, 17.0225600731029, 17.0233934115189, 17.0242267499349, 
17.0250600883509, 17.0258934267669, 17.0267267651829, 17.027560103599, 
17.028393442015, 17.029226780431, 17.030060118847, 17.030893457263, 
17.031726795679, 17.0325601340951, 17.0333934725111, 17.0342268109271, 
17.0350601493431, 17.0358934877591, 17.0367268261751, 17.0375601645912, 
17.0383935030072, 17.0392268414232, 17.0400601798392, 17.0408935182552, 
17.0417268566712, 17.0425601950873, 17.0433935335033, 17.0442268719193, 
17.0450602103353, 17.0458935487513, 17.0467268871673, 17.0475602255834, 
17.0483935639994, 17.0492269024154, 17.0500602408314, 17.0508935792474, 
17.0517269176635, 17.0525602560795, 17.0533935944955, 17.0542269329115, 
17.0550602713275, 17.0558936097435, 17.0567269481596, 17.0575602865756, 
17.0583936249916, 17.0592269634076, 17.0600603018236, 17.0608936402396, 
17.0617269786557, 17.0625603170717, 17.0633936554877, 17.0642269939037, 
17.0650603323197, 17.0658936707357, 17.0667270091518, 17.0675603475678, 
17.0683936859838, 17.0692270243998, 17.0700603628158, 17.0708937012318, 
17.0717270396479, 17.0725603780639, 17.0733937164799, 17.0742270548959, 
17.0750603933119, 17.0758937317279, 17.076727070144, 17.07756040856, 
17.078393746976, 17.079227085392, 17.080060423808, 17.080893762224, 
17.0817271006401, 17.0825604390561, 17.0833937774721, 17.0842271158881, 
17.0850604543041, 17.0858937927201, 17.0867271311362, 17.0875604695522, 
17.0883938079682, 17.0892271463842, 17.0900604848002, 17.0908938232162, 
17.0917271616323, 17.0925605000483, 17.0933938384643, 17.0942271768803, 
17.0950605152963, 17.0958938537123, 17.0967271921284, 17.0975605305444, 
17.0983938689604, 17.0992272073764, 17.1000605457924, 17.1008938842084, 
17.1017272226245, 17.1025605610405, 17.1033938994565, 17.1042272378725, 
17.1050605762885, 17.1058939147046, 17.1067272531206, 17.1075605915366, 
17.1083939299526, 17.1092272683686, 17.1100606067846, 17.1108939452007, 
17.1117272836167, 17.1125606220327, 17.1133939604487, 17.1142272988647, 
17.1150606372807, 17.1158939756968, 17.1167273141128, 17.1175606525288, 
17.1183939909448, 17.1192273293608, 17.1200606677768, 17.1208940061929, 
17.1217273446089, 17.1225606830249, 17.1233940214409, 17.1242273598569, 
17.1250606982729, 17.125894036689, 17.126727375105, 17.127560713521, 
17.128394051937, 17.129227390353, 17.130060728769, 17.1308940671851, 
17.1317274056011, 17.1325607440171, 17.1333940824331, 17.1342274208491, 
17.1350607592651, 17.1358940976812, 17.1367274360972, 17.1375607745132, 
17.1383941129292, 17.1392274513452, 17.1400607897612, 17.1408941281773, 
17.1417274665933, 17.1425608050093, 17.1433941434253, 17.1442274818413, 
17.1450608202573, 17.1458941586734, 17.1467274970894, 17.1475608355054, 
17.1483941739214, 17.1492275123374, 17.1500608507534, 17.1508941891695, 
17.1517275275855, 17.1525608660015, 17.1533942044175, 17.1542275428335, 
17.1550608812496, 17.1558942196656, 17.1567275580816, 17.1575608964976, 
17.1583942349136, 17.1592275733296, 17.1600609117457, 17.1608942501617, 
17.1617275885777, 17.1625609269937, 17.1633942654097, 17.1642276038257, 
17.1650609422418, 17.1658942806578, 17.1667276190738, 17.1675609574898, 
17.1683942959058, 17.1692276343218, 17.1700609727379, 17.1708943111539, 
17.1717276495699, 17.1725609879859, 17.1733943264019, 17.1742276648179, 
17.175061003234, 17.17589434165, 17.176727680066, 17.177561018482, 
17.178394356898, 17.179227695314, 17.1800610337301, 17.1808943721461, 
17.1817277105621, 17.1825610489781, 17.1833943873941, 17.1842277258101, 
17.1850610642262, 17.1858944026422, 17.1867277410582, 17.1875610794742, 
17.1883944178902, 17.1892277563062, 17.1900610947223, 17.1908944331383, 
17.1917277715543, 17.1925611099703, 17.1933944483863, 17.1942277868023, 
17.1950611252184, 17.1958944636344, 17.1967278020504, 17.1975611404664, 
17.1983944788824, 17.1992278172984, 17.2000611557145, 17.2008944941305, 
17.2017278325465, 17.2025611709625, 17.2033945093785, 17.2042278477945, 
17.2050611862106, 17.2058945246266, 17.2067278630426, 17.2075612014586, 
17.2083945398746, 17.2092278782907, 17.2100612167067, 17.2108945551227, 
17.2117278935387, 17.2125612319547, 17.2133945703707, 17.2142279087868, 
17.2150612472028, 17.2158945856188, 17.2167279240348, 17.2175612624508, 
17.2183946008668, 17.2192279392829, 17.2200612776989, 17.2208946161149, 
17.2217279545309, 17.2225612929469, 17.2233946313629, 17.224227969779, 
17.225061308195, 17.225894646611, 17.226727985027, 17.227561323443, 
17.228394661859, 17.2292280002751, 17.2300613386911, 17.2308946771071, 
17.2317280155231, 17.2325613539391, 17.2333946923551, 17.2342280307712, 
17.2350613691872, 17.2358947076032, 17.2367280460192, 17.2375613844352, 
17.2383947228512, 17.2392280612673, 17.2400613996833, 17.2408947380993, 
17.2417280765153, 17.2425614149313, 17.2433947533473, 17.2442280917634, 
17.2450614301794, 17.2458947685954, 17.2467281070114, 17.2475614454274, 
17.2483947838434, 17.2492281222595, 17.2500614606755, 17.2508947990915, 
17.2517281375075, 17.2525614759235, 17.2533948143395, 17.2542281527556, 
17.2550614911716, 17.2558948295876, 17.2567281680036, 17.2575615064196, 
17.2583948448356, 17.2592281832517, 17.2600615216677, 17.2608948600837, 
17.2617281984997, 17.2625615369157, 17.2633948753318, 17.2642282137478, 
17.2650615521638, 17.2658948905798, 17.2667282289958, 17.2675615674118, 
17.2683949058279, 17.2692282442439, 17.2700615826599, 17.2708949210759, 
17.2717282594919, 17.2725615979079, 17.273394936324, 17.27422827474, 
17.275061613156, 17.275894951572, 17.276728289988, 17.277561628404, 
17.2783949668201, 17.2792283052361, 17.2800616436521, 17.2808949820681, 
17.2817283204841, 17.2825616589001, 17.2833949973162, 17.2842283357322, 
17.2850616741482, 17.2858950125642, 17.2867283509802, 17.2875616893962, 
17.2883950278123, 17.2892283662283, 17.2900617046443, 17.2908950430603, 
17.2917283814763, 17.2925617198923, 17.2933950583084, 17.2942283967244, 
17.2950617351404, 17.2958950735564, 17.2967284119724, 17.2975617503884, 
17.2983950888045, 17.2992284272205, 17.3000617656365, 17.3008951040525, 
17.3017284424685, 17.3025617808845, 17.3033951193006, 17.3042284577166, 
17.3050617961326, 17.3058951345486, 17.3067284729646, 17.3075618113806, 
17.3083951497967, 17.3092284882127, 17.3100618266287, 17.3108951650447, 
17.3117285034607, 17.3125618418768, 17.3133951802928, 17.3142285187088, 
17.3150618571248, 17.3158951955408, 17.3167285339568, 17.3175618723729, 
17.3183952107889, 17.3192285492049, 17.3200618876209, 17.3208952260369, 
17.3217285644529, 17.322561902869, 17.323395241285, 17.324228579701, 
17.325061918117, 17.325895256533, 17.326728594949, 17.3275619333651, 
17.3283952717811, 17.3292286101971, 17.3300619486131, 17.3308952870291, 
17.3317286254451, 17.3325619638612), value = c(8.83893229166667, 
8.84375, 8.844921875, 8.84544270833333, 8.84544270833333, 8.84518229166667, 
8.84466145833333, 8.84231770833333, 8.83984375, 8.83854166666667, 
8.83645833333333, 8.83489583333333, 8.83359375, 8.83958333333333, 
8.84088541666667, 8.84205729166667, 8.8375, 8.83802083333333, 
8.83815104166667, 8.838671875, 8.83802083333333, 8.840234375, 
8.83958333333333, 8.84088541666667, 8.83880208333333, 8.83984375, 
8.83958333333333, 8.8421875, 8.84622395833333, 8.849609375, 8.84856770833333, 
8.84596354166667, 8.84114583333333, 8.83841145833333, 8.83645833333333, 
8.83697916666667, 8.83984375, 8.84479166666667, 8.84791666666667, 
8.847265625, 8.84244791666667, 8.84192708333333, 8.84231770833333, 
8.84140625, 8.840234375, 8.83776041666667, 8.839453125, 8.83958333333333, 
8.84010416666667, 8.843359375, 8.84557291666667, 8.84661458333333, 
8.843359375, 8.84192708333333, 8.84205729166667, 8.8421875, 8.84114583333333, 
8.83958333333333, 8.83958333333333, 8.84140625, 8.83997395833333, 
8.83841145833333, 8.83841145833333, 8.840234375, 8.84088541666667, 
8.83932291666667, 8.84192708333333, 8.843359375, 8.84609375, 
8.84635416666667, 8.84921875, 8.84596354166667, 8.842578125, 
8.83684895833333, 8.83697916666667, 8.84309895833333, 8.84635416666667, 
8.84830729166667, 8.84427083333333, 8.84505208333333, 8.84557291666667, 
8.84791666666667, 8.85104166666667, 8.85247395833333, 8.85078125, 
8.84596354166667, 8.84205729166667, 8.84114583333333, 8.84140625, 
8.84049479166667, 8.83997395833333, 8.84036458333333, 8.84231770833333, 
8.84440104166667, 8.84934895833333, 8.84934895833333, 8.84557291666667, 
8.838671875, 8.83854166666667, 8.83828125, 8.83606770833333, 
8.83515625, 8.83854166666667, 8.83997395833333, 8.83958333333333, 
8.83828125, 8.841796875, 8.84283854166667, 8.842578125, 8.84453125, 
8.84401041666667, 8.84466145833333, 8.83841145833333, 8.838671875, 
8.84322916666667, 8.84661458333333, 8.85065104166667, 8.84947916666667, 
8.84830729166667, 8.84518229166667, 8.84231770833333, 8.844140625, 
8.84375, 8.84361979166667, 8.84322916666667, 8.841015625, 8.840234375, 
8.83984375, 8.84127604166667, 8.84309895833333, 8.84322916666667, 
8.84557291666667, 8.84622395833333, 8.84544270833333, 8.84348958333333, 
8.84075520833333, 8.84270833333333, 8.8484375, 8.85221354166667, 
8.84973958333333, 8.84361979166667, 8.839453125, 8.841015625, 
8.84153645833333, 8.84296875, 8.84283854166667, 8.84544270833333, 
8.84635416666667, 8.84609375, 8.84479166666667, 8.84440104166667, 
8.84453125, 8.846875, 8.84921875, 8.85104166666667, 8.85013020833333, 
8.84674479166667, 8.84674479166667, 8.84609375, 8.84466145833333, 
8.84375, 8.84375, 8.84713541666667, 8.845703125, 8.84739583333333, 
8.84778645833333, 8.84973958333333, 8.84700520833333, 8.84479166666667, 
8.84700520833333, 8.84830729166667, 8.85104166666667, 8.850390625, 
8.8484375, 8.84947916666667, 8.85247395833333, 8.85442708333333, 
8.85403645833333, 8.85221354166667, 8.85390625, 8.851953125, 
8.851171875, 8.85013020833333, 8.855078125, 8.86002604166667, 
8.86588541666667, 8.86471354166667, 8.86432291666667, 8.86341145833333, 
8.86184895833333, 8.859765625, 8.85807291666667, 8.85872395833333, 
8.85625, 8.855078125, 8.85677083333333, 8.86041666666667, 8.8625, 
8.86015625, 8.85885416666667, 8.86015625, 8.86263020833333, 8.86653645833333, 
8.86341145833333, 8.85911458333333, 8.854296875, 8.85885416666667, 
8.86458333333333, 8.86653645833333, 8.86809895833333, 8.87122395833333, 
8.87473958333333, 8.87434895833333, 8.87200520833333, 8.870703125, 
8.86796875, 8.86640625, 8.86549479166667, 8.86666666666667, 8.86640625, 
8.86796875, 8.86692708333333, 8.86549479166667, 8.8609375, 8.86158854166667, 
8.86328125, 8.86458333333333, 8.864453125, 8.86458333333333, 
8.86627604166667, 8.862890625, 8.862890625, 8.86432291666667, 
8.86341145833333, 8.86145833333333, 8.85950520833333, 8.8625, 
8.864453125, 8.86263020833333, 8.86184895833333, 8.86106770833333, 
8.86341145833333, 8.86471354166667, 8.86575520833333, 8.86692708333333, 
8.86458333333333, 8.86080729166667, 8.85572916666667, 8.858203125, 
8.85716145833333, 8.85533854166667, 8.854296875, 8.86041666666667, 
8.87005208333333, 8.86875, 8.86223958333333, 8.856640625, 8.859765625, 
8.86119791666667, 8.85572916666667, 8.8515625, 8.84895833333333, 
8.8484375, 8.84856770833333, 8.85182291666667, 8.85299479166667, 
8.84908854166667, 8.84635416666667, 8.84765625, 8.84986979166667, 
8.84739583333333, 8.84947916666667, 8.85403645833333, 8.86158854166667, 
8.86197916666667, 8.86028645833333, 8.8578125, 8.85794270833333, 
8.858984375, 8.85872395833333, 8.85911458333333, 8.85638020833333, 
8.85299479166667, 8.85234375, 8.85416666666667, 8.86041666666667, 
8.85859375, 8.85546875, 8.85143229166667, 8.85026041666667, 8.849609375, 
8.84700520833333, 8.84765625, 8.84609375, 8.84765625, 8.848046875, 
8.85104166666667, 8.85598958333333, 8.85755208333333, 8.8578125, 
8.85338541666667, 8.85026041666667, 8.85052083333333, 8.85234375, 
8.85286458333333, 8.85403645833333, 8.85533854166667, 8.85729166666667, 
8.85338541666667, 8.85013020833333, 8.84869791666667, 8.85013020833333, 
8.8515625, 8.85169270833333, 8.846875, 8.840234375, 8.83854166666667, 
8.84166666666667, 8.84895833333333, 8.85286458333333, 8.85481770833333, 
8.848828125, 8.84518229166667, 8.84427083333333, 8.85013020833333, 
8.85247395833333, 8.85390625, 8.8515625, 8.84830729166667, 8.84869791666667, 
8.85013020833333, 8.85247395833333, 8.84791666666667, 8.84205729166667, 
8.84036458333333, 8.84322916666667, 8.84505208333333, 8.84270833333333, 
8.841796875, 8.84401041666667, 8.84713541666667, 8.84778645833333, 
8.84583333333333, 8.84244791666667, 8.83997395833333, 8.83841145833333, 
8.84153645833333, 8.84296875, 8.84388020833333, 8.83997395833333, 
8.83580729166667, 8.83333333333333, 8.83971354166667, 8.84283854166667, 
8.844140625, 8.836328125, 8.834765625, 8.834375, 8.840234375, 
8.842578125, 8.84609375, 8.84674479166667, 8.848046875, 8.84908854166667, 
8.84674479166667, 8.841015625, 8.836328125, 8.836328125, 8.83645833333333, 
8.83697916666667, 8.83815104166667, 8.84166666666667, 8.84153645833333, 
8.84114583333333, 8.84479166666667, 8.84700520833333, 8.84791666666667, 
8.84270833333333, 8.84309895833333, 8.84427083333333, 8.84388020833333, 
8.84166666666667, 8.833984375, 8.83385416666667, 8.83177083333333, 
8.83268229166667, 8.83684895833333, 8.83932291666667, 8.84205729166667, 
8.84088541666667, 8.837890625, 8.84010416666667, 8.84231770833333, 
8.84778645833333, 8.84674479166667, 8.84192708333333, 8.84127604166667, 
8.84153645833333, 8.844140625, 8.84322916666667, 8.83932291666667, 
8.83723958333333, 8.83893229166667, 8.84557291666667, 8.84700520833333, 
8.84440104166667, 8.84166666666667, 8.84088541666667, 8.83958333333333, 
8.84153645833333, 8.84375, 8.85182291666667, 8.851953125, 8.85364583333333, 
8.85104166666667, 8.84908854166667, 8.8453125, 8.841015625, 8.84036458333333, 
8.84440104166667, 8.85078125, 8.859765625, 8.86158854166667, 
8.86041666666667, 8.858203125, 8.86067708333333, 8.86614583333333, 
8.86901041666667, 8.873828125, 8.87643229166667, 8.87408854166667, 
8.87278645833333, 8.87578125, 8.88515625, 8.88919270833333, 8.89075520833333, 
8.89557291666667, 8.90286458333333, 8.91380208333333, 8.92317708333333, 
8.93138020833333, 8.93697916666667, 8.94244791666667, 8.949609375, 
8.958203125, 8.96875, 8.98372395833333, 8.996484375, 9.01041666666667, 
9.02018229166667, 9.035546875, 9.050390625, 9.06979166666667, 
9.08658854166667, 9.10846354166667, 9.1265625, 9.151171875, 9.17057291666667, 
9.192578125, 9.21510416666667, 9.24231770833333, 9.27278645833333, 
9.30104166666667, 9.32760416666667, 9.35286458333333, 9.3796875, 
9.40989583333333, 9.43567708333333, 9.46341145833333, 9.494140625, 
9.53125, 9.56692708333333, 9.60377604166667, 9.644921875, 9.67994791666667, 
9.7109375, 9.74205729166667, 9.77825520833333, 9.81471354166667, 
9.848046875, 9.88515625, 9.92161458333333, 9.95572916666667, 
9.98723958333333, 10.0186197916667, 10.0555989583333, 10.090625, 
10.1231770833333, 10.150390625, 10.173828125, 10.1985677083333, 
10.22265625, 10.2481770833333, 10.2680989583333, 10.285546875, 
10.2967447916667, 10.306640625, 10.3169270833333, 10.3319010416667, 
10.34375, 10.349609375, 10.351171875, 10.3553385416667, 10.3583333333333, 
10.3572916666667, 10.3537760416667, 10.3514322916667, 10.3506510416667, 
10.3477864583333, 10.33515625, 10.3174479166667, 10.2983072916667, 
10.28515625, 10.2708333333333, 10.249609375, 10.2208333333333, 
10.1932291666667, 10.1661458333333, 10.1401041666667, 10.117578125, 
10.094921875, 10.0670572916667, 10.0345052083333, 10.0061197916667, 
9.985546875, 9.95768229166667, 9.921875, 9.88645833333333, 9.85559895833333, 
9.83177083333333, 9.80338541666667, 9.77330729166667, 9.73580729166667, 
9.70208333333333, 9.66744791666667, 9.64166666666667, 9.61614583333333, 
9.58684895833333, 9.55455729166667, 9.525390625, 9.49947916666667, 
9.47174479166667, 9.43411458333333, 9.40260416666667, 9.373828125, 
9.34778645833333, 9.32447916666667, 9.29739583333333, 9.27760416666667, 
9.255078125, 9.23567708333333, 9.21302083333333, 9.18854166666667, 
9.16692708333333, 9.14921875, 9.13294270833333, 9.11666666666667, 
9.09973958333333, 9.08606770833333, 9.07213541666667, 9.05794270833333, 
9.04466145833333, 9.03203125, 9.017578125, 9.00234375, 8.990234375, 
8.985546875, 8.97890625, 8.96979166666667, 8.96041666666667, 
8.95247395833333, 8.94388020833333, 8.93684895833333, 8.93059895833333, 
8.92161458333333, 8.90807291666667, 8.90221354166667, 8.90546875, 
8.90520833333333, 8.89674479166667, 8.887109375, 8.88411458333333, 
8.88567708333333, 8.88333333333333, 8.8796875, 8.8734375, 8.870703125, 
8.87005208333333, 8.86966145833333, 8.87057291666667, 8.87109375, 
8.8703125, 8.86471354166667, 8.85963541666667, 8.86236979166667, 
8.86236979166667, 8.86171875, 8.854296875, 8.85338541666667, 
8.85234375, 8.850390625)), row.names = 20200:20800, class = "data.frame")

Edit: Attempted use of regSmooth package

Here I have applied the github.com/domwoolf/regSmooth package to a small section of data shown in close up on the baseline. The red line is the smoothed values, the black line is the original data, and the vertical dotted blue and green lines are local minima and maxima respectively, as detected using the turning point function from the 'pastecs' package. Notice that the smoothed line has large dips before and after the major peak at ~12.4 on the x axis. These aren't present in the raw data and they lead to misplaced local minima when calculated using the smoothed line. Also, in the more or less flat region between x=12.05 and x=12.15, several minima and maxima are identified, indicating insufficient smoothing in that region. The smoothing was accomplished using the regSmoothAuto function, which automates optimization of the lambda value. Perhaps a manual assignment of lambda would improve the smoothing achieved here?

plot(x,y, type = 'l',ylim=c(floor(min(y)), ceiling(mean(tail(fid_df)$value))))  
regsmoothed_fid<-regSmoothAuto(x,y)  
lines (regsmoothed_fid$yhat ~ x, col="red", lw=2)

tp<-pastecs::turnpoints(y)
abline(v=x[tp$peaks],lty=2, col='blue')
  abline(v=x[tp$pits],lty=2,col='green')

Edit: pracma::whittaker()

The pracma::whittaker() smoothing function applied to a 1 minute subset of my data produces similar artifacts to those mentioned above for the regSmoothAuto() function. i.e large dips before and after major peaks.

Answer 1

This article (A Perfect Smoother; Paul H. C. Eilers) describes an algorithm extremely well suited to your needs. I have written an R package with it that you can get using

devtools::install_github('domwoolf/regSmooth')

Using this on your example data

fid$yhat = regSmooth(fid$rt, fid$value, 3e-5, d = 3)$yhat

we get

zooming on on the baseline, it looks like this

fid.plot = fid[fid$rt<17.1, ]
plot(fid.plot$rt, fid.plot$value, col='blue', type='l')
lines(fid.plot$rt, fid.plot$yhat, col='red')

---

Update Since your x values are uniformly spaced, a simpler version of this algorithm can also be used. This simpler version (which assumes uniformly monotonically incremented x values) is available in the package pracma. In this version, the default value of lambda provides reasonable smoothing.

library(pracma)
fid$yhat = whittaker(fid$value)

gives us

Answer 2

Here is an example of using the RcppRoll rolling mean function, to get the mean of every 5 records in a sliding window.

https://cran.r-project.org/web/packages/RcppRoll/index.html

#install.packages("RcppRoll")
library(RcppRoll)
df$value2 <- roll_mean(df$value, n=5, fill=0)

Example

df<-data.frame(row=1:1000, value=sample(c(1:100), 1000, replace=T))
df$value2<-roll_mean(df$value, n=10, fill=0)
library(tidyverse)
df %>% ggplot(aes(x=row)) + geom_line(aes(y=value), color="blue", alpha=0.5)+geom_line(aes(y=value2), color="red")