Hexycode
Hexycode

Reputation: 458

MongoDB aggregation returns empty array with NodeJS

I want to create a script which is taking the average of the Volume for last 7(for example) days. I'm stuck with aggregation stages since first stage I need to take Date for last 7 days and in second stage calculate Average of Volume

Package list:

Node-schedule - */1 * * * * (Runs the script every minute)

Binance API - Taking data from them.

Screenshot for showcasing how the document looks like in MongoDB.

enter image description here

Aggregation part of the Code.

const average = await dbo.collection(symbol).aggregate([{
                '$match': {
                    'Date': { '$gte': new Date((new Date().getTime() - (7 * 24 * 60 * 60 * 1000))) }
                },
            },
            {
                '$group': {
                    _id: null,
                    'Volume': { '$avg': '$Volume' }
                },
            }
        ]).toArray();

This code returns me an empty array in terminal like this > []

Full Code here.

const { MongoClient } = require('mongodb');
const schedule = require('node-schedule');
const fetch = require("node-fetch");
const symbols = ["ADABTC", "AEBTC", "AIONBTC", "ALGOBTC", "ARDRBTC"];

//a descriptive name helps your future self and others understand code easier
const getBTCData = async symbol => { //make this function accept the current symbol
    //async/await lets us write this much nicer and with less nested indents
    let data = await fetch(`https://api.binance.com/api/v3/klines?symbol=${symbol}&interval=30m&limit=1`).then(res => res.json());
    const btcusdtdata = data.map(d => {
        return {
            Open: parseFloat(d[1]),
            High: parseFloat(d[2]),
            Low: parseFloat(d[3]),
            Close: parseFloat(d[4]),
            Volume: parseFloat(d[5]),
            Timespan: 30,
        }
    });
    console.log(btcusdtdata);
    saveToDatebase(symbol, btcusdtdata);
    //recursive functions are complicated, we can get rid of it here
    //by moving the responsibility to the caller
};

//helper function for an awaitable timeout
const sleep = ms => new Promise(res => setTimeout(res, ms));

const j = schedule.scheduleJob('*/1 * * * *', async() => {
    //expand this function to be responsible for looping the data
    for (let symbol of symbols) {
        //we can pass symbol to getBTCData instead of making it
        //responsible for figuring out which symbol it should get
        await getBTCData(symbol);
        await sleep(8000);
    }
});

//make this a helper function so `saveToDatabase()` isn't also responsible for it
const getDateTime = () => {
    let today = new Date();
    let date = today.getFullYear() + '-' + (today.getMonth() + 1) + '-' + today.getDate();
    let time = today.getHours() + ":" + today.getMinutes() + ":" + today.getSeconds();
    return date + ' ' + time;
};

const saveToDatebase = async(symbol, BTCdata) => {
    try {
        const url = 'mongodb://username:password@ipadress:port/dbname';
        let dateTime = getDateTime();
        let db = await MongoClient.connect(url, { useUnifiedTopology: true });
        const dbo = db.db('Crypto');
        const myobj = Object.assign({ Name: symbol, Date: dateTime }, BTCdata[0]);
        await dbo.collection(symbol).insertOne(myobj);
        const average = await dbo.collection(symbol).aggregate([{
                '$match': {
                    'Date': { '$gte': new Date((new Date().getTime() - (7 * 24 * 60 * 60 * 1000))) }
                },
            },
            {
                '$group': {
                    _id: null,
                    'Volume': { '$avg': '$Volume' }
                },
            }
        ]).toArray();
        console.log('1 document inserted');
        console.log(average);
        db.close();
    } catch (e) {
        console.error(e)
    }
};

EDIT1 If I delete $match part my script is working and I receive average of Volume.

Screenshot of terminal after success try without $match

enter image description here

EDIT2

According to the last answer I understand that I need to change Date format from string to object, but I really can't get how I can do it in this part?

 const getDateTime = () => {
    let today = new Date();
    let date = today.getFullYear() + '-' + (today.getMonth() + 1) + '-' + today.getDate();
    let time = today.getHours() + ":" + today.getMinutes() + ":" + today.getSeconds();
    return date + ' ' + time;
};

EDIT3

After editing the Date format I receive a Document in MongoDB in strange Date format like - Date:2020-07-20T13:24:02.390+00:00

Code here:

const getDateTime = () => {
    let today = new Date();
    let date = today.getFullYear() + '-' + (today.getMonth() + 1) + '-' + today.getDate();
    let time = today.getHours() + ":" + today.getMinutes() + ":" + today.getSeconds();
    return new Date();
};

Upvotes: 0

Views: 873

Answers (1)

Manuel Spigolon
Manuel Spigolon

Reputation: 12870

The problem is on the Date field format.

The getDateTime function returns a string so Mongo is managing the field as a string not as a Date object so the $gte check will compare string not dates.

You should change the function to getDateTime = () => new Date(). Mongo will manage the date correctly storing in UTF Timezone.

Tring to query a date-string in the $match field would be really difficult.

Edit:

To update the typing just:

const getDateTime = () => {
    return new Date();
};

Upvotes: 3

Related Questions