Printing out result in 0/1 KnapSack (Recursive Brute Force)

Question

public static int KnapSack(int capacity, Item[] items, int numItems) {
    if (numItems == 0 || capacity == 0)
        return 0;
    if (items[numItems-1].weight > capacity)
        return KnapSack(capacity, items, numItems-1);
    else {
        int took = items[numItems-1].value + KnapSack(capacity - items[numItems-1].weight, items, numItems-1);
        int left = KnapSack(capacity, items, numItems-1);
        return Math.max(took, left);
    }     
}  

So I have a working 0/1 recursive brute force algorithm working for the KnapSack problem. I was wondering of what would be an approach to print out the working solution (i.e the items collected into the knapsack from the set of items). I've tried a number of things such as adding into a list and trying to keep track of what things I have added, but none of worked out either do to implementation or design problem. So I came here for some help, thanks!

Answer 1

I know its been already answered, just adding code versions in c#, just for reference (for simplified knapsack you may refer to: How do I solve the 'classic' knapsack algorithm recursively?):

version 1. Solving using Dynamic Programming (similar to wiki) - Bottom Up (tabulated approach)

version 2. Solving using Dynamic Programming but Top to bottom (Memoization - lazy)

version 3. Recursive (just recursive solution without using overlapped sub problems or optimal substructure properties that enable to use DP)

version 4. Brute force (going through all combinations) - captures max values and value indices in fields (this._valueIndices, this._maxValue)(refer to tests2 - which should show the caller and usage of these captured values)

References: http://en.wikipedia.org/wiki/Knapsack_problem, How do I solve the 'classic' knapsack algorithm recursively?

Tabular - DP - Version (O(nW) - pseudo polynomial) - O(nW) memory - Bottom up

public int Knapsack_0_1_DP_Tabular_Bottom_Up(int[] weights, int[] values, int maxWeight)
        {
            this.ValidataInput_Knapsack_0_1(weights, values, maxWeight);
            int[][] DP_Memoization_Max_Value_Cache = new int[values.Length + 1][];
            for (int i = 0; i <= values.Length; i++)
            {
                DP_Memoization_Max_Value_Cache[i] = new int[maxWeight + 1];
                for (int j = 0; j <= maxWeight; j++)
                {
                    DP_Memoization_Max_Value_Cache[i][j] = 0; //yes, its default - 
                }
            }
            /// f(i, w) = f(i-1, w) if Wi > w
            ///         Or, max (f(i-1, w), f(i-1, w-Wi) + Vi
            ///         Or 0 if i < 0 
            for(int i = 1; i<=values.Length; i++)
            {
                for(int w = 1; w <= maxWeight; w++)
                {
                    //below code can be refined - intentional as i just want it
                    //look similar to other 2 versions (top_to_bottom_momoization
                    //and recursive_without_resuing_subproblem_solution
                    int maxValueWithoutIncludingCurrentItem =
                            DP_Memoization_Max_Value_Cache[i - 1][w];
                    if (weights[i - 1] > w)
                    {
                        DP_Memoization_Max_Value_Cache[i][w] = maxValueWithoutIncludingCurrentItem;
                    }
                    else
                    {
                        int maxValueByIncludingCurrentItem =
                            DP_Memoization_Max_Value_Cache[i - 1][w - weights[i - 1]]
                            + values[i-1];
                        int overAllMax = maxValueWithoutIncludingCurrentItem;
                        if(maxValueByIncludingCurrentItem > overAllMax)
                        {
                            overAllMax = maxValueByIncludingCurrentItem;   
                        }
                        DP_Memoization_Max_Value_Cache[i][w] = overAllMax;
                    }
                }
            }
            return DP_Memoization_Max_Value_Cache[values.Length][maxWeight];
        }

DP - Memoization - Top to Bottom - Lazy evaluation

/// <summary>
        /// f(i, w) = f(i-1, w) if Wi > w
        ///         Or, max (f(i-1, w), f(i-1, w-Wi) + Vi
        ///         Or 0 if i < 0 
        /// </summary>
        int Knapsack_0_1_DP_Memoization_Top_To_Bottom_Lazy_Recursive(int[] weights, int[] values,
            int index, int weight, int?[][] DP_Memoization_Max_Value_Cache)
        {
            if (index < 0)
            {
                return 0;
            }
            Debug.Assert(weight >= 0);
#if DEBUG
            if ((index == 0) || (weight == 0))
            {
                Debug.Assert(DP_Memoization_Max_Value_Cache[index][weight] == 0);
            }
#endif
            //value is cached, so return
            if(DP_Memoization_Max_Value_Cache[index][weight] != null)
            {
                return DP_Memoization_Max_Value_Cache[index][weight].Value;
            }
            Debug.Assert(index > 0);
            Debug.Assert(weight > 0);
            int maxValueWithoutIncludingCurrentItem = this.Knapsack_0_1_DP_Memoization_Top_To_Bottom_Lazy_Recursive
                (weights, values, index - 1, weight, DP_Memoization_Max_Value_Cache);
            if (weights[index-1] > weight)
            {
                DP_Memoization_Max_Value_Cache[index][weight] = maxValueWithoutIncludingCurrentItem;
                //current item weight is more, so we cant include - so, just return
                return maxValueWithoutIncludingCurrentItem;
            }
            int overallMaxValue = maxValueWithoutIncludingCurrentItem;
            int maxValueIncludingCurrentItem = this.Knapsack_0_1_DP_Memoization_Top_To_Bottom_Lazy_Recursive
                (weights, values, index - 1, weight - weights[index-1],
                    DP_Memoization_Max_Value_Cache) + values[index - 1];
            if(maxValueIncludingCurrentItem > overallMaxValue)
            {
                overallMaxValue = maxValueIncludingCurrentItem;
            }
            DP_Memoization_Max_Value_Cache[index][weight] = overallMaxValue;
            return overallMaxValue;
        }

And public method to call (for details on callers, refer to below units tests)

public int Knapsack_0_1_DP_Tabular_Bottom_Up(int[] weights, int[] values, int maxWeight)
        {
            this.ValidataInput_Knapsack_0_1(weights, values, maxWeight);
            int[][] DP_Memoization_Max_Value_Cache = new int[values.Length + 1][];
            for (int i = 0; i <= values.Length; i++)
            {
                DP_Memoization_Max_Value_Cache[i] = new int[maxWeight + 1];
                for (int j = 0; j <= maxWeight; j++)
                {
                    DP_Memoization_Max_Value_Cache[i][j] = 0; //yes, its default - 
                }
            }
            /// f(i, w) = f(i-1, w) if Wi > w
            ///         Or, max (f(i-1, w), f(i-1, w-Wi) + Vi
            ///         Or 0 if i < 0 
            for(int i = 1; i<=values.Length; i++)
            {
                for(int w = 1; w <= maxWeight; w++)
                {
                    //below code can be refined - intentional as i just want it
                    //look similar to other 2 versions (top_to_bottom_momoization
                    //and recursive_without_resuing_subproblem_solution
                    int maxValueWithoutIncludingCurrentItem =
                            DP_Memoization_Max_Value_Cache[i - 1][w];
                    if (weights[i - 1] > w)
                    {
                        DP_Memoization_Max_Value_Cache[i][w] = maxValueWithoutIncludingCurrentItem;
                    }
                    else
                    {
                        int maxValueByIncludingCurrentItem =
                            DP_Memoization_Max_Value_Cache[i - 1][w - weights[i - 1]]
                            + values[i-1];
                        int overAllMax = maxValueWithoutIncludingCurrentItem;
                        if(maxValueByIncludingCurrentItem > overAllMax)
                        {
                            overAllMax = maxValueByIncludingCurrentItem;   
                        }
                        DP_Memoization_Max_Value_Cache[i][w] = overAllMax;
                    }
                }
            }
            return DP_Memoization_Max_Value_Cache[values.Length][maxWeight];
        }

Recursive - with DP sub problems - but without reusing overlapping sub problems (this should depict how it is easier to change recursive version to DP Top to bottom (Memoization version)

public int Knapsack_0_1_OverlappedSubPromblems_OptimalSubStructure(int[] weights, int[] values, int maxWeight)
        {
            this.ValidataInput_Knapsack_0_1(weights, values, maxWeight);
            int v = this.Knapsack_0_1_OverlappedSubPromblems_OptimalSubStructure_Recursive(weights, values, index: weights.Length-1, weight: maxWeight);
            return v;
        }
        /// <summary>
        /// f(i, w) = f(i-1, w) if Wi > w
        ///         Or, max (f(i-1, w), f(i-1, w-Wi) + Vi
        ///         Or 0 if i < 0 
        /// </summary>
        int Knapsack_0_1_OverlappedSubPromblems_OptimalSubStructure_Recursive(int[] weights, int[] values, int index, int weight)
        {
            if (index < 0)
            {
                return 0;
            }
            Debug.Assert(weight >= 0);
            int maxValueWithoutIncludingCurrentItem = this.Knapsack_0_1_OverlappedSubPromblems_OptimalSubStructure_Recursive(weights,
                values, index - 1, weight);
            if(weights[index] > weight)
            {
                //current item weight is more, so we cant include - so, just return
                return maxValueWithoutIncludingCurrentItem;
            }
            int overallMaxValue = maxValueWithoutIncludingCurrentItem;
            int maxValueIncludingCurrentItem = this.Knapsack_0_1_OverlappedSubPromblems_OptimalSubStructure_Recursive(weights,
                values, index - 1, weight - weights[index]) + values[index];
            if(maxValueIncludingCurrentItem > overallMaxValue)
            {
                overallMaxValue = maxValueIncludingCurrentItem;
            }
            return overallMaxValue;
        }

Brute Force (going through all combinations)

private int _maxValue = int.MinValue;
        private int[] _valueIndices = null;
        public void Knapsack_0_1_BruteForce_2_Power_N(int[] weights, int[] values, int maxWeight)
        {
            this.ValidataInput_Knapsack_0_1(weights, values, maxWeight);
            this._maxValue = int.MinValue;
            this._valueIndices = null;
            this.Knapsack_0_1_BruteForce_2_Power_N_Rcursive(weights, values, maxWeight, 0, 0, 0, new List<int>());
        }
        private void Knapsack_0_1_BruteForce_2_Power_N_Rcursive(int[] weights, int[] values, int maxWeight, int index, int currentWeight, int currentValue, List<int> currentValueIndices)
        {
            if(currentWeight > maxWeight)
            {
                return;
            }
            if(currentValue > this._maxValue)
            {
                this._maxValue = currentValue;
                this._valueIndices = currentValueIndices.ToArray();
            }
            if(index == weights.Length)
            {
                return;
            }
            Debug.Assert(index < weights.Length);
            var w = weights[index];
            var v = values[index];
            //see if the current value, conributes to max value
            currentValueIndices.Add(index);
            Knapsack_0_1_BruteForce_2_Power_N_Rcursive(weights, values, maxWeight, index + 1, currentWeight + w,
                currentValue + v, currentValueIndices);
            currentValueIndices.Remove(index);
            //see if current value, does not contribute to max value
            Knapsack_0_1_BruteForce_2_Power_N_Rcursive(weights, values, maxWeight, index + 1, currentWeight, currentValue,
                currentValueIndices);
        }

Unit Tests 1

[TestMethod]
        public void Knapsack_0_1_Tests()
        {
            int[] benefits = new int[] { 60, 100, 120 };
            int[] weights = new int[] { 10, 20, 30 };
            this.Knapsack_0_1_BruteForce_2_Power_N(weights, values: benefits, maxWeight: 50);
            Assert.IsTrue(this._maxValue == 220);
            int v = this.Knapsack_0_1_OverlappedSubPromblems_OptimalSubStructure(weights, 
                values: benefits, maxWeight: 50);
            Assert.IsTrue(v == 220);
            v = this.Knapsack_0_1_DP_Memoization_Top_To_Bottom_Lazy(weights,
                values: benefits, maxWeight: 50);
            Assert.IsTrue(v == 220);
            v = this.Knapsack_0_1_DP_Tabular_Bottom_Up(weights,
                values: benefits, maxWeight: 50);
            Assert.IsTrue(v == 220);
            benefits = new int[] { 3, 4, 5, 8, 10 };
            weights = new int[] { 2, 3, 4, 5, 9 };
            this.Knapsack_0_1_BruteForce_2_Power_N(weights, values: benefits, maxWeight: 20);
            Assert.IsTrue(this._maxValue == 26);
            v = this.Knapsack_0_1_OverlappedSubPromblems_OptimalSubStructure(weights, values: benefits,
                maxWeight: 20);
            Assert.IsTrue(v == 26);
            v = this.Knapsack_0_1_DP_Memoization_Top_To_Bottom_Lazy(weights,
                values: benefits, maxWeight: 20);
            Assert.IsTrue(v == 26);
            v = this.Knapsack_0_1_DP_Tabular_Bottom_Up(weights,
                values: benefits, maxWeight: 20);
            Assert.IsTrue(v == 26);
            benefits = new int[] { 3, 4, 5, 6};
            weights = new int[] { 2, 3, 4, 5 };
            this.Knapsack_0_1_BruteForce_2_Power_N(weights, values: benefits, maxWeight: 5);
            Assert.IsTrue(this._maxValue == 7);
            v = this.Knapsack_0_1_OverlappedSubPromblems_OptimalSubStructure(weights, values: benefits,
                maxWeight: 5);
            Assert.IsTrue(v == 7);
            v = this.Knapsack_0_1_DP_Memoization_Top_To_Bottom_Lazy(weights,
                values: benefits, maxWeight: 5);
            Assert.IsTrue(v == 7);
            v = this.Knapsack_0_1_DP_Tabular_Bottom_Up(weights,
                values: benefits, maxWeight: 5);
            Assert.IsTrue(v == 7);
        }

Unit Tests 2

[TestMethod]
        public void Knapsack_0_1_Brute_Force_Tests()
        {
            int[] benefits = new int[] { 60, 100, 120 };
            int[] weights = new int[] { 10, 20, 30 };
            this.Knapsack_0_1_BruteForce_2_Power_N(weights, values: benefits, maxWeight: 50);
            Assert.IsTrue(this._maxValue == 220);
            Assert.IsTrue(this._valueIndices.Contains(1));
            Assert.IsTrue(this._valueIndices.Contains(2));
            Assert.IsTrue(this._valueIndices.Length == 2);
            this._maxValue = int.MinValue;
            this._valueIndices = null;
            benefits = new int[] { 3, 4, 5, 8, 10 };
            weights = new int[] { 2, 3, 4, 5, 9 };
            this.Knapsack_0_1_BruteForce_2_Power_N(weights, values: benefits, maxWeight: 20);
            Assert.IsTrue(this._maxValue == 26);
            Assert.IsTrue(this._valueIndices.Contains(0));
            Assert.IsTrue(this._valueIndices.Contains(2));
            Assert.IsTrue(this._valueIndices.Contains(3));
            Assert.IsTrue(this._valueIndices.Contains(4));
            Assert.IsTrue(this._valueIndices.Length == 4);
        }

Answer 2

To track the items taken, try something like:

public static int KnapSack(int capacity, Item[] items, int numItems, ArrayList<Integer> taken) {
    if (numItems == 0 || capacity == 0)
        return 0;
    if (items[numItems-1].weight > capacity)
        return KnapSack(capacity, items, numItems-1, taken);
    else {
        final int preTookSize = taken.size();
        final int took = items[numItems-1].value + KnapSack(capacity - items[numItems-1].weight, items, numItems-1, taken);

        final int preLeftSize = taken.size();
        final int left = KnapSack(capacity, items, numItems-1, taken);

        if (took > left) {
            if (taken.size() > preLeftSize)
                taken.removeRange(preLeftSize, taken.size());
            taken.add(Integer.valueOf(numItems - 1));
            return took;
        }
        else {
            if (preLeftSize > preTookSize)
                taken.removeRange(preTookSize, preLeftSize);
            return left;
        }
    }     
}

This may not be the most efficient, but I think it should work.

(For efficiency you might try pre-allocating the taken ArrayList to be "big enough" such that no allocations need to happen during the recursive looping.)