How to reduce memory usage on UICollectionView with large quantity of items?

Question

I'm trying to use a UICollectionView instead of a UITableView to display a very large number of items (>15000) and it seems the UICollectionView pre-allocates a pixel buffer for the entire contentView size required by the collection. Depending on the item sizes, the simulator shows up to 6.75GB of memory required.

I was hoping the collection view, based on its protocol being very similar to UITableView, wouldn't allocate any pixel buffer and rely solely on the cells' backing/rendering.

I'm using a storyboard file to define the collection view and cell, both have Opaque = false. I've looked at numerous articles on Stack Overflow, most have to do with memory leaks so I'm a little stumped as to how to fix the issue.

For the curious, here's the entire code base (other than the Storyboard):

MyCollectionViewCell.swift

import UIKit    
class MyCollectionViewCell: UICollectionViewCell {
    static let identifier = "myCellIdentifier"
}

UIColor+Random.swift

import UIKit
extension UIColor {
    static func randomColor() -> UIColor {
        let red = CGFloat(drand48())
        let green = CGFloat(drand48())
        let blue = CGFloat(drand48())
        return UIColor(red: red, green: green, blue: blue, alpha: 1.0)
    }
}

ViewController.swift

import UIKit
class ViewController: UIViewController, UICollectionViewDataSource, UICollectionViewDelegate, UICollectionViewDelegateFlowLayout {

    func numberOfSections(in collectionView: UICollectionView) -> Int {
        return 10000
    }

    func collectionView(_ collectionView: UICollectionView, numberOfItemsInSection section: Int) -> Int {
        return 30000
    }

    func collectionView(_ collectionView: UICollectionView, cellForItemAt indexPath: IndexPath) -> UICollectionViewCell {
        let cell = collectionView.dequeueReusableCell(withReuseIdentifier: MyCollectionViewCell.identifier, for: indexPath) as! MyCollectionViewCell
        cell.backgroundColor = UIColor.randomColor()
        return cell
    }


    @IBOutlet weak var collectionView: UICollectionView!
    override func viewDidLoad() {
        super.viewDidLoad()
        // Do any additional setup after loading the view, typically from a nib.
    }
}

Memory usage under certain use cases:

Answer 1

The UICollectionView is suited to manage a limited number of sections. Using very large amount of sections and items will invariably lead to large amounts of memory being used as UIKit attempts to keep track of the positioning of every elements in the sections to scale the contentView appropriately.

Instead, create a custom UICollectionViewLayout to manage the relationship between the very large number of items, where they should be positioned and coordinate with the UICollectionViewDataSource to map a relatively small set of items as a "window" to a much larger set.

For example, say UICollectionView items are as per the question about 100x100; UIKit would render approximately 80~100 on the screen at once. Assuming UICollectionView caches a few entries on all side as the user scrolls, having 1024 items in a "cache" to rotate through should be more than sufficient. UICollectionView has no issue at all managing 1 section with 1024 items.

Next, using a custom UICollectionViewLayout, define a custom contentViewSize large enough to hold all the items. UICollectionView will be inquiring the layout about which items are within a visible rectangle via layoutAttributesForElements(in rect: CGRect) -> [UICollectionViewLayoutAttributes]?

Make sure to return a new set of item's every time the collection view is inquiring. Using a 1024 items cache start with item index 0 for the first element displayed on screen, 1 for second, 2 for 3rd, etc. Coordinating with your UICollectionViewDataSource, every time a coordinate in your layout is associated with a item index, inform the data source about the true section/item that should be associated with the index associated.

First, lets define a protocol & data source to map the true size of the large data source to something reasonable UICollectionView can handle.

LargeDataSourceCoordinator.swift:

import UIKit

protocol LargeDataSourceProtocol {
    func largeNumberOfSections() -> Int
    func largeNumberOfItems(in section: Int) -> Int
    func largeNumberToCollectionViewCacheSize() -> Int
    func associateLargeIndexPath(_ largeIndexPath: IndexPath) -> IndexPath
}

class LargeDataSourceCoordinator: NSObject, UICollectionViewDataSource, LargeDataSourceProtocol {

    var cachedMapEntries: [IndexPath: IndexPath] = [:]
    var rotatingCacheIndex: Int = 0

    func largeNumberToCollectionViewCacheSize() -> Int {
        return 1024 // arbitrary number, increase if rendering issues are visible like cells not appearing when scrolling
    }

    func largeNumberOfSections() -> Int {
        // To do: implement logic to find the number of sections
        return 10000 // simplified arbitrary number for sake of demo
    }

    func largeNumberOfItems(in section: Int) -> Int {
        // To do: implement logic to find the number of items in each section
        return 30000  // simplified arbitrary number for sake of demo
    }

    func associateLargeIndexPath(_ largeIndexPath: IndexPath) -> IndexPath {
        for existingPath in cachedMapEntries where existingPath.value == largeIndexPath {
            return existingPath.key
        }
        let collectionViewIndexPath = IndexPath(item: rotatingCacheIndex, section: 0)
        cachedMapEntries[collectionViewIndexPath] = largeIndexPath
        rotatingCacheIndex = (rotatingCacheIndex + 1) % self.largeNumberToCollectionViewCacheSize()
        return collectionViewIndexPath
    }

    func numberOfSections(in collectionView: UICollectionView) -> Int {
        return 1
    }

    func collectionView(_ collectionView: UICollectionView, numberOfItemsInSection section: Int) -> Int {
        return self.largeNumberToCollectionViewCacheSize()
    }

    func collectionView(_ collectionView: UICollectionView, cellForItemAt indexPath: IndexPath) -> UICollectionViewCell {
        let cell = MyCollectionViewCell.dequeue(from: collectionView, for: indexPath)
        guard let largeIndexPath = cachedMapEntries[indexPath] else { return cell }

        // retrieve the data at largeIndexPath.section, largeIndexPath.item
        // configure cell accordingly
        cell.addDebugText("section: \(largeIndexPath.section)\nitem: \(largeIndexPath.item)")
        return cell
    }
}

Next, let's create UICollectionViewLayout to help position and coordinate with the data source. For sake of simplicity, cell use fixed size of 100x100, each section is displayed immediately after the other with cells packed to the left on each row.

LargeDataSourceLayout.swift:

import UIKit

class LargeDataSourceLayout: UICollectionViewLayout {

    let cellSize = CGSize(width: 100, height: 100)

    var cellsPerRow: CGFloat {
        guard let collectionView = self.collectionView else { return 1.0 }
        return (collectionView.frame.size.width / cellSize.width).rounded(.towardZero)
    }

    var cacheNumberOfItems: [Int] = []
    private func refreshNumberOfItemsCache() {
        guard
            let largeDataSource = self.collectionView?.dataSource as? LargeDataSourceProtocol
            else { return }
        cacheNumberOfItems.removeAll()
        for section in 0 ..< largeDataSource.largeNumberOfSections() {
            let itemsInSection: Int = largeDataSource.largeNumberOfItems(in: section)
            cacheNumberOfItems.append(itemsInSection)
        }
    }

    var cacheRowsPerSection: [Int] = []
    private func refreshRowsPerSection() {
        let itemsPerRow = Float(self.cellsPerRow)
        cacheRowsPerSection.removeAll()
        for section in 0 ..< cacheNumberOfItems.count {
            let numberOfItems = Float(cacheNumberOfItems[section])
            let numberOfRows = (numberOfItems / itemsPerRow).rounded(.awayFromZero)
            cacheRowsPerSection.append(Int(numberOfRows))
        }
    }

    override var collectionViewContentSize: CGSize {
        // To do: update logic as per your requirements
        refreshNumberOfItemsCache()
        refreshRowsPerSection()
        let totalRows = cacheRowsPerSection.reduce(0, +)
        return CGSize(width: self.cellsPerRow * cellSize.width,
                      height: CGFloat(totalRows) * cellSize.height)
    }

    override func layoutAttributesForItem(at indexPath: IndexPath) -> UICollectionViewLayoutAttributes? {
        // To do: implement logic to compute the attributes for a specific item
        return nil
    }

    private func originForRow(_ row: Int) -> CGFloat {
        return CGFloat(row) * cellSize.height
    }

    private func pathsInRow(_ row: Int) -> [IndexPath] {
        let itemsPerRow = Int(self.cellsPerRow)
        var subRowIndex = row
        for section in 0 ..< cacheRowsPerSection.count {
            let rowsInSection = cacheRowsPerSection[section]
            if subRowIndex < rowsInSection {
                let firstItem = subRowIndex * itemsPerRow
                let lastItem = min(cacheNumberOfItems[section],firstItem+itemsPerRow) - 1
                var paths: [IndexPath] = []
                for item in firstItem ... lastItem {
                    paths.append(IndexPath(item: item, section: section))
                }
                return paths
            } else {
                guard rowsInSection <= subRowIndex else { return [] }
                subRowIndex -= rowsInSection
            }
        }
        // if caches are properly updated, we should never reach here
        return []
    }

    override func layoutAttributesForElements(in rect: CGRect) -> [UICollectionViewLayoutAttributes]? {
        guard let largeDataSource = self.collectionView?.dataSource as? LargeDataSourceProtocol else { return nil }

        let firstRow = max(0,Int((rect.minY / cellSize.height).rounded(.towardZero)))
        var row = firstRow
        var attributes: [UICollectionViewLayoutAttributes] = []
        repeat {
            let originY = originForRow(row)
            if originY > rect.maxY {
                return attributes
            }

            var originX: CGFloat = 0.0
            for largeIndexPath in pathsInRow(row) {
                let indexPath = largeDataSource.associateLargeIndexPath(largeIndexPath)
                let itemAttribute = UICollectionViewLayoutAttributes(forCellWith: indexPath)
                itemAttribute.frame = CGRect(x: originX, y: originY, width: cellSize.width, height: cellSize.height)
                attributes.append(itemAttribute)
                originX += cellSize.width
            }

            row += 1
        } while true
    }
}

There are a few things happening in there, and there's tons of room for optimization based on use-case, but the concept is there. For sake of completion, below is associated code and app preview.

MyCollectionViewCell.swift:

import UIKit

class MyCollectionViewCell: UICollectionViewCell {

    static let identifier = "MyCollectionViewCell"

    static func dequeue(from collectionView: UICollectionView, for indexPath: IndexPath) -> MyCollectionViewCell {
        let cell = collectionView.dequeueReusableCell(withReuseIdentifier: identifier, for: indexPath) as? MyCollectionViewCell ?? MyCollectionViewCell()
        cell.contentView.backgroundColor = UIColor.random()
        return cell
    }

    override func prepareForReuse() {
        super.prepareForReuse()
        removeDebugLabel()
    }

    private func removeDebugLabel() {
        self.contentView.subviews.first?.removeFromSuperview()
    }

    func addDebugText(_ text: String) {
        removeDebugLabel()
        let debugLabel = UILabel(frame: CGRect(x: 0, y: 0, width: 100, height: 100))
        debugLabel.text = text
        debugLabel.numberOfLines = 2
        debugLabel.font = UIFont.systemFont(ofSize: UIFont.smallSystemFontSize)
        debugLabel.textColor = UIColor.black
        debugLabel.textAlignment = .center
        self.contentView.addSubview(debugLabel)
    }
}

UIColor+random.swift:

import UIKit

extension UIColor {
    static func random() -> UIColor {
        //random color
        let hue = CGFloat(arc4random() % 256) / 256.0
        let saturation = (CGFloat(arc4random() % 128) / 256.0) + 0.5 // 0.5 to 1.0, away from white
        let brightness = (CGFloat(arc4random() % 128) / 256.0 ) + 0.5 // 0.5 to 1.0, away from black
        return UIColor(hue: hue, saturation: saturation, brightness: brightness, alpha: 1.0)
    }
}