Google App Script - Error when Mapping - TypeError: Cannot read property 'map' of undefined

Question

With the guidance of the community I've recently learned here about BatchUpdating background colors of a given google sheet.

I've attempted to apply this to my actual sheet but run into the error code. TypeError: Cannot read property 'map' of undefined

The code in my test sheet which works without issue is here:

var TestArray = Sheets.Spreadsheets.get("1eAq-RbtrCSMRPZ0p7XIpG3vd29yL-3SQ3D3JGyiUhKg", {
   ranges:"TestBackgroundSheet!A1:AD39", fields:"sheets(data(rowData(values(effectiveFormat.backgroundColor))))"
 });
 
 var backgroundColors = TestArray["sheets"][0]["data"][0]["rowData"]
                      .map(row => row["values"]
                      .map(value => value["effectiveFormat"]["backgroundColor"])); 

I then copied and pasted this code into my actual project. In an attempt to troubleshoot I even copied the exact Sheet from my test project into the actual project. The code is listed here:

var TestArray = Sheets.Spreadsheets.get("1pcIKNUFmkk0d-UGg1sXl5xbsJC2WhocIHpM3et-CMgo", {
   ranges:"TestBackgroundSheet!A1:AD39", fields:"sheets(data(rowData(values(effectiveFormat.backgroundColor))))"
 });
 
 var backgroundColors = TestArray["sheets"][0]["data"][0]["rowData"]
                      .map(row => row["values"]
                      .map(value => value["effectiveFormat"]["backgroundColor"]));

As you can see the code is exactly the same except for a different SheetID. Despite this i continue to get the TypeError: Cannot read property 'map' of undefined Error.

The error code references line 172 which is .map(value => value["effectiveFormat"]["backgroundColor"]));

Edit:

Edit #2:

With inclusion of Rafa's Code I get this error code -> SyntaxError: Unexpected token , in JSON at position 1 (line 177, file "macros")

Included code is below.

var TestArray = Sheets.Spreadsheets.get("1pcIKNUFmkk0d-UGg1sXl5xbsJC2WhocIHpM3et-CMgo", {
      ranges:"TestBackgroundSheet!A1:AD39",
      fields:"sheets(data(rowData(values(effectiveFormat.backgroundColor))))"
    });
     
    var rowData = TestArray["sheets"][0]["data"][0]["rowData"]
    .map(row => row.getValues()).toString()
    
    var backgroundColors = JSON.parse("[" + rowData + "]")
    .map(value => {
         let v = value["effectiveFormat"]
         return v ? v["backgroundColor"] : null
         })

Edit #3:

Logger of RowData with below code

for (var x = 0; x < 40; x++) {
      Logger.log(x + JSON.stringify(TestArray["sheets"][0]["data"][0]["rowData"][x]));
    }

I can see the issue is in the first four lines of Logger statements. While the empty rows in the "Testing Sheet" returns full rows of White background color (RGB {"green":1,"red":1,"blue":1}) In the "Reat Sheet" I get an empty object for 3 of the first 4 lines. I've include the first four lines out output below. I'm not sure why this occurs.

Actual:

[20-10-21 08:53:23:591 EDT] 0{}
[20-10-21 08:53:23:593 EDT] 1{"values":[{},{"effectiveFormat":{"backgroundColor":{"green":1,"blue":1,"red":1}}}]}
[20-10-21 08:53:23:595 EDT] 2{}
[20-10-21 08:53:23:597 EDT] 3{}
[20-10-21 08:53:23:600 EDT] 4{"values":[{"effectiveFormat":{"backgroundColor":{"green":1,"red":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"blue":1,"red":1}}},{},{"effectiveFormat":{"backgroundColor":{"blue":1,"green":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"red":1,"blue":1}}},{},{"effectiveFormat":{"backgroundColor":{"red":1,"green":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"green":1,"red":1}}},{},{"effectiveFormat":{"backgroundColor":{"blue":1,"green":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"red":1,"blue":1}}},{},{"effectiveFormat":{"backgroundColor":{"red":1,"green":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"green":1,"red":1}}},{},{"effectiveFormat":{"backgroundColor":{"red":1,"green":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"blue":1,"red":1}}},{},{"effectiveFormat":{"backgroundColor":{"red":1,"blue":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"green":1,"blue":1}}},{},{"effectiveFormat":{"backgroundColor":{"blue":1,"green":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"green":1,"red":1}}},{},{"effectiveFormat":{"backgroundColor":{"red":1,"blue":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"red":1,"blue":1}}},{},{"effectiveFormat":{"backgroundColor":{"green":1,"red":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"blue":1,"red":1}}}]}

Testing:

[20-10-21 05:53:14:167 PDT] 0{"values":[{"effectiveFormat":{"backgroundColor":{"green":1,"red":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"blue":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"red":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"green":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"red":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"blue":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"blue":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"blue":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"red":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"green":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"blue":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"green":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"green":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"green":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"red":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"green":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"blue":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"blue":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"green":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"green":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"blue":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"blue":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"red":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"blue":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"red":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"green":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"green":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"blue":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"red":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"red":1,"blue":1}}}]}
[20-10-21 05:53:14:170 PDT] 1{"values":[{"effectiveFormat":{"backgroundColor":{"red":1,"green":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"red":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"blue":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"red":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"blue":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"green":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"green":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"red":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"red":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"blue":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"red":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"blue":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"blue":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"blue":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"blue":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"red":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"red":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"green":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"blue":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"red":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"red":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"green":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"green":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"green":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"red":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"red":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"green":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"green":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"blue":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"blue":1,"red":1}}}]}
[20-10-21 05:53:14:193 PDT] 2{"values":[{"effectiveFormat":{"backgroundColor":{"green":1,"blue":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"red":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"red":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"red":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"red":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"red":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"blue":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"green":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"green":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"blue":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"green":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"green":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"green":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"green":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"blue":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"blue":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"red":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"green":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"red":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"red":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"blue":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"red":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"blue":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"blue":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"red":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"green":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"green":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"green":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"green":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"blue":1,"red":1}}}]}
[20-10-21 05:53:14:195 PDT] 3{"values":[{"effectiveFormat":{"backgroundColor":{"red":1,"blue":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"blue":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"blue":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"green":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"green":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"red":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"red":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"blue":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"red":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"green":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"green":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"blue":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"blue":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"green":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"red":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"green":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"blue":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"green":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"green":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"green":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"red":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"red":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"red":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"blue":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"red":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"green":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"blue":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"red":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"green":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"green":1,"red":1}}}]}
[20-10-21 05:53:14:198 PDT] 4{"values":[{"effectiveFormat":{"backgroundColor":{"red":1,"green":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"red":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"blue":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"red":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"blue":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"blue":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"red":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"blue":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"red":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"green":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"green":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"red":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"blue":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"red":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"green":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"blue":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"red":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"green":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"blue":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"green":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"blue":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"green":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"blue":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"blue":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"red":1,"green":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"green":1,"red":1,"blue":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"green":1,"red":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"red":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"red":1,"green":1}}},{"effectiveFormat":{"backgroundColor":{"blue":1,"green":1,"red":1}}}]}

Answer 1

Answer:

Not all elements of row["values"] have data - and so you can't run .map(value => value["effectiveFormat"]["backgroundColor"]) when that row doesn't have data.

More Information:

You've got your data from the API using

Sheets.Spreadsheets.get("1eAq-RbtrCSMRPZ0p7XIpG3vd29yL-3SQ3D3JGyiUhKg")

and filtered it accordingly with your field mask. The data you get as a response will be everything in the sheet - even the cells that do not have data for background colour. As a result of this, you can not map each row like this, as you will try and reference the effectiveFormat element where it simply doesn't exist.

Fix:

You can use the ternary operator to solve this; if the element value["effectiveFormat"] doesn't exist you can simply return null:

var rowData = TestArray["sheets"][0]["data"][0]["rowData"]
    .map(row => row.getValues()).toString()

var backgroundColors = JSON.parse("[" + rowData + "]")
    .map(value => {
        let v = value["effectiveFormat"]
        return v ? v["backgroundColor"] : null
    })

NB: The API also returns functions within the JSON objects in the response which you can use in Apps Script. This comes in handy because referencing row["values"] directly can return the object rather than the data:

console.log(TestArray["sheets"][0]["data"][0]["rowData"]
            .map(row => row["values"]))

yields:

[ 
  { 
    setPivotTable: [Function],
    getDataSourceTable: [Function],
    getDataValidation: [Function],
    getEffectiveValue: [Function],
    setNote: [Function],
    setFormattedValue: [Function],
    getTextFormatRuns: [Function],
    setUserEnteredFormat: [Function],
    toString: [Function],
    getFormattedValue: [Function],
    setEffectiveFormat: [Function],
    effectiveFormat: [Object],
    setDataSourceFormula: [Function],
    getPivotTable: [Function],
    setUserEnteredValue: [Function],
    setDataValidation: [Function],
    setDataSourceTable: [Function],
    getUserEnteredFormat: [Function],
    setEffectiveValue: [Function],
    getEffectiveFormat: [Function],
    getHyperlink: [Function],
    getNote: [Function],
    setHyperlink: [Function],
    getUserEnteredValue: [Function],
    setTextFormatRuns: [Function],
    getDataSourceFormula: [Function] 
  },
  ...
]

The toString() call after the first .map(row => row.getValues()) gets past this before putting the data back into the second mapping function.

References:

• Conditional (ternary) operator - JavaScript | MDN