Hubbry Logo
List of knapsack problemsList of knapsack problemsMain
Open search
List of knapsack problems
Community hub
List of knapsack problems
logo
7 pages, 0 posts
0 subscribers
Be the first to start a discussion here.
Be the first to start a discussion here.
List of knapsack problems
List of knapsack problems
from Wikipedia

The knapsack problem is one of the most studied problems in combinatorial optimization, with many real-life applications. For this reason, many special cases and generalizations have been examined.[1][2]

Common to all versions are a set of n items, with each item having an associated profit pj and weight wj. The binary decision variable xj is used to select the item. The objective is to pick some of the items, with maximal total profit, while obeying that the maximum total weight of the chosen items must not exceed W. Generally, these coefficients are scaled to become integers, and they are almost always assumed to be positive.

The knapsack problem in its most basic form:

maximize
subject to

Direct generalizations

[edit]

One common variant is that each item can be chosen multiple times. The bounded knapsack problem specifies, for each item j, an upper bound uj (which may be a positive integer, or infinity) on the number of times item j can be selected:

maximize
subject to
integral for all j

The unbounded knapsack problem (sometimes called the integer knapsack problem) does not put any upper bounds on the number of times an item may be selected:

maximize
subject to
integral for all j

The unbounded variant was shown to be NP-complete in 1975 by Lueker.[3] Both the bounded and unbounded variants admit an FPTAS (essentially the same as the one used in the 0-1 knapsack problem).

If the items are subdivided into k classes denoted , and exactly one item must be taken from each class, we get the multiple-choice knapsack problem:

maximize
subject to
for all
for all and all

If for each item the profit and weight are equal, we get the subset sum problem (often the corresponding decision problem is given instead):

maximize
subject to

If we have n items and m knapsacks with capacities , we get the multiple knapsack problem:

maximize
subject to for all
for all
for all and all

As a special case of the multiple knapsack problem, when the profits are equal to weights and all bins have the same capacity, we can have multiple subset sum problem.

Quadratic knapsack problem:

maximize
subject to
for all

Set-Union Knapsack Problem:

SUKP is defined by Kellerer et al[2] (on page 423) as follows:

Given a set of items and a set of so-called elements , each item corresponds to a subset of the element set . The items have non-negative profits , , and the elements have non-negative weights , . The total weight of a set of items is given by the total weight of the elements of the union of the corresponding element sets. The objective is to find a subset of the items with total weight not exceeding the knapsack capacity and maximal profit.

Multiple constraints

[edit]

If there is more than one constraint (for example, both a volume limit and a weight limit, where the volume and weight of each item are not related), we get the multiple-constrained knapsack problem, multidimensional knapsack problem, or m-dimensional knapsack problem. (Note, "dimension" here does not refer to the shape of any items.) This has 0-1, bounded, and unbounded variants; the unbounded one is shown below.

maximize
subject to for all
, integer for all

The 0-1 variant (for any fixed ) was shown to be NP-complete around 1980 and more strongly, has no FPTAS unless P=NP.[4][5]

The bounded and unbounded variants (for any fixed ) also exhibit the same hardness.[6]

For any fixed , these problems do admit a pseudo-polynomial time algorithm (similar to the one for basic knapsack) and a PTAS.[2]

Knapsack-like problems

[edit]

If all the profits are 1, we will try to maximize the number of items which would not exceed the knapsack capacity:

maximize
subject to

If we have a number of containers (of the same size), and we wish to pack all n items in as few containers as possible, we get the bin packing problem, which is modelled by having indicator variables container i is being used:

minimize
subject to

The cutting stock problem is identical to the bin packing problem, but since practical instances usually have far fewer types of items, another formulation is often used. Item j is needed Bj times, each "pattern" of items which fit into a single knapsack have a variable, xi (there are m patterns), and pattern i uses item j bij times:

minimize
subject to for all
for all

If, to the multiple choice knapsack problem, we add the constraint that each subset is of size n and remove the restriction on total weight, we get the assignment problem, which is also the problem of finding a maximal bipartite matching:

maximize
subject to for all
for all
for all and all

In the Maximum Density Knapsack variant there is an initial weight , and we maximize the density of selected items which do not violate the capacity constraint: [7]

maximize
subject to

Although less common than those above, several other knapsack-like problems exist, including:

  • Nested knapsack problem
  • Collapsing knapsack problem
  • Nonlinear knapsack problem
  • Inverse-parametric knapsack problem

The last three of these are discussed in Kellerer et al's reference work, Knapsack Problems.[2]

References

[edit]
Revisions and contributorsEdit on WikipediaRead on Wikipedia
from Grokipedia
The knapsack problem is a classic combinatorial optimization challenge in which a decision-maker selects a subset of items—each characterized by a weight and a value—to include in a knapsack of fixed capacity, aiming to maximize the total value while ensuring the total weight does not exceed the capacity limit. This NP-hard problem, with applications in resource allocation, scheduling, and logistics, serves as the foundation for numerous variants that adapt the core model to more complex real-world scenarios by introducing constraints like multiple selections, fractional items, or additional resource dimensions. The traces its origins to 1897, when George B. Mathews first formulated a version of it. In , Bellman introduced dynamic programming techniques to solve the 0-1 variant. Further advancements in the 1960s by Peter C. Gilmore and Ronald E. Gomory explored efficient dynamic programming solutions. By 1976, the problem found applications in , notably in the work of and . Key variants include the 0-1 knapsack problem, where each item can be chosen at most once, solvable via dynamic programming in O(nW) for n items and capacity W; the unbounded knapsack problem, allowing unlimited repetitions of items; and the bounded knapsack problem, permitting up to a specified number of each item. Other notable extensions are the fractional knapsack problem, which permits partial item inclusion and is greedily solvable in O(n log n) time, and the multiple knapsack problem, involving assignment to several knapsacks with potentially varying capacities for applications like vehicle loading or processor scheduling. Further generalizations encompass the multidimensional knapsack problem, incorporating multiple constraint types (e.g., weight and volume); the multiple-choice knapsack problem, where items are grouped into classes with exactly one selection per class; and specialized forms like the quadratic knapsack problem with nonlinear objectives or the online knapsack problem, where items arrive sequentially without full prior knowledge. These variants, often addressed through techniques such as branch-and-bound, approximation algorithms, or heuristics, highlight the knapsack family's versatility in modeling diverse optimization challenges across fields like , , and project selection.

Introduction

Definition and Basic Formulation

The is a fundamental in , where the goal is to select a subset of items, each characterized by a weight and a profit, to maximize the total profit without exceeding a given knapsack capacity. The standard mathematical formulation of the basic 0/1 knapsack problem, which serves as the foundation for many variants, is to maximize the objective function j=1npjxj\sum_{j=1}^{n} p_j x_j subject to the capacity constraint j=1nwjxjc\sum_{j=1}^{n} w_j x_j \leq c, where xj{0,1}x_j \in \{0,1\} for each item j=1,,nj = 1, \dots, n. Here, pjp_j represents the non-negative profit of item jj, wjw_j denotes its non-negative weight, cc is the capacity of the knapsack, and the decision variable xj=1x_j = 1 if item jj is selected and xj=0x_j = 0 otherwise. The objective function seeks to maximize the total profit from selected items, while the single linear constraint ensures the total weight does not exceed the capacity. Key terminology includes the set of nn items, each with associated profit pjp_j and wjw_j, the knapsack capacity cc, and the focus on under resource limitations. The 0/1 is NP-hard, though it admits a dynamic programming solution with O(nc)O(n c). A standard dynamic programming approach fills a table dpdp representing the maximum profit using the first ii items with capacity up to jj, using the recurrence dp=max(dp[i1],dp[i1][jwi]+pi)dp = \max(dp[i-1], dp[i-1][j - w_i] + p_i) if jwij \geq w_i, else dp=dp[i1]dp = dp[i-1], with base case dp{{grok:render&&&type=render_inline_citation&&&citation_id=0&&&citation_type=wikipedia}} = 0 for all jj. The following illustrates this bottom-up implementation:

for i from 1 to n: for j from 0 to c: if j < w_i: dp[i][j] = dp[i-1][j] else: dp[i][j] = max(dp[i-1][j], dp[i-1][j - w_i] + p_i) return dp[n][c]

for i from 1 to n: for j from 0 to c: if j < w_i: dp[i][j] = dp[i-1][j] else: dp[i][j] = max(dp[i-1][j], dp[i-1][j - w_i] + p_i) return dp[n][c]

This algorithm computes the optimal solution efficiently when cc is not too large.

Historical Development and Applications

The traces its mathematical origins to the late 19th century, where early formulations appeared in studies of number partitioning and subset sums. A seminal early reference is G. B. Mathews' 1896 paper on the partition of numbers, which addressed problems closely related to selecting subsets with constrained sums. The term "knapsack problem" itself is attributed to Tobias Dantzig, a mathematician and father of operations research pioneer , who reportedly coined it in the early 20th century to evoke the metaphor of packing a traveler's backpack. The problem gained formal structure in operations research during the 1950s, as computational methods emerged for discrete optimization. George B. Dantzig explored discrete-variable extremum problems, including knapsack-like formulations, in his 1957 work, proposing greedy heuristics for unbounded variants. Concurrently, Richard Bellman introduced dynamic programming principles in the same decade, laying groundwork for exact solutions to bounded cases. Key milestones in the 1960s included the development of branch-and-bound techniques, which systematically enumerated solutions while pruning infeasible branches to improve efficiency on larger instances. By 1979, Michael R. Garey and David S. Johnson cataloged the 0/1 knapsack problem as NP-complete in their influential guide to computational complexity, solidifying its role as a canonical hard optimization problem. In the 1970s, dynamic programming algorithms evolved into practical tools, offering pseudopolynomial-time solutions that scaled better for moderate-sized instances compared to earlier enumeration methods. This progression from manual partitioning approaches to algorithmic frameworks marked a shift toward broader applicability in computing. Knapsack problems model resource allocation in diverse fields, such as cargo loading, where carriers select goods to maximize revenue without exceeding weight limits. In budget planning and project selection, they aid management decisions by optimizing investments under fixed capital constraints. Additionally, the subset sum variant underpins cryptography, notably in the Merkle-Hellman knapsack cryptosystem introduced in 1978, which relies on the problem's hardness for secure encryption.

Classical Variants

0/1 Knapsack Problem

The 0/1 knapsack problem is a fundamental combinatorial optimization problem in which a set of nn items, each characterized by a weight wi>0w_i > 0 and a profit pi>0p_i > 0 for i=1,,ni = 1, \dots, n, must be selected such that the total weight does not exceed a given knapsack capacity W>0W > 0, while maximizing the total profit. The decision variables are binary, xi{0,1}x_i \in \{0, 1\}, where xi=1x_i = 1 indicates that item ii is included and xi=0x_i = 0 otherwise. The problem is formally stated as: maxi=1npixi\max \sum_{i=1}^n p_i x_i subject to i=1nwixiW,xi{0,1}i=1,,n.\sum_{i=1}^n w_i x_i \leq W, \quad x_i \in \{0, 1\} \quad \forall i = 1, \dots, n. This variant enforces that each item can be chosen at most once, distinguishing it from relaxations that permit multiple or fractional inclusions. The decision version of the 0/1 knapsack problem—determining whether there exists a subset of items with total profit at least some value PP and total weight at most WW—is NP-complete. Despite this, the optimization problem admits a pseudo-polynomial time algorithm using dynamic programming, running in O(nW)O(nW) time and O(nW)O(nW) space, which is efficient when weights are small relative to nn. The approach builds a table dpdp representing the maximum profit achievable using the first ii items with capacity exactly ww. The recurrence relation is: dp={max(dp[i1],dp[i1][wwi]+pi)if wwi,dp[i1]otherwise,dp = \begin{cases} \max(dp[i-1], \, dp[i-1][w - w_i] + p_i) & \text{if } w \geq w_i, \\ dp[i-1] & \text{otherwise}, \end{cases} with base cases dp{{grok:render&&&type=render_inline_citation&&&citation_id=0&&&citation_type=wikipedia}} = 0 for all ww and dp{{grok:render&&&type=render_inline_citation&&&citation_id=0&&&citation_type=wikipedia}} = 0 for all ii. The optimal solution is maxw=0Wdp\max_{w=0}^W dp. Space can be optimized to O(W)O(W) by using a one-dimensional array and iterating backward over weights. For cases where exact solutions are infeasible due to large WW, a fully polynomial-time approximation scheme (FPTAS) exists that computes a solution within (1ϵ)(1 - \epsilon) of the optimal profit in O(n2/ϵ)O(n^2 / \epsilon) time, for any fixed ϵ>0\epsilon > 0. This scheme scales polynomially in the input size and 1/ϵ1/\epsilon, making it practical for high-precision approximations. Example. Consider n=3n=3 items with weights w=(2,3,4)w = (2, 3, 4), profits p=(3,4,5)p = (3, 4, 5), and capacity W=5W=5. The dynamic programming table yields an optimal selection of items 1 and 2 (weights 2 and 3, total weight 5, total profit 7), outperforming other combinations like items 2 and 3 (total weight 7 > 5, infeasible) or item 3 alone (profit 5).

Unbounded Knapsack Problem

The Unbounded Knapsack Problem (UKP) is a problem in which an unlimited number of copies of each item type may be included in the knapsack, distinguishing it from variants that impose restrictions on item quantities. Formally, given a set of nn item types, each characterized by a positive weight wiw_i and profit pip_i for i=1,,ni = 1, \dots, n, and a knapsack capacity WW, the goal is to determine non-negative integers xix_i that maximize the total profit i=1npixi\sum_{i=1}^n p_i x_i subject to the capacity constraint i=1nwixiW\sum_{i=1}^n w_i x_i \leq W. This formulation arises in applications such as and cutting stock problems, where repetition of resource types is feasible. The standard exact solution employs dynamic programming, leveraging the problem's . Define dpdp as the maximum profit achievable with capacity exactly ww, for w=0,,Ww = 0, \dots, W, initialized with dp{{grok:render&&&type=render_inline_citation&&&citation_id=0&&&citation_type=wikipedia}} = 0 and dp=dp = -\infty otherwise (or 0 for non-negative handling). For each item i=1i = 1 to nn, and for each capacity w=wiw = w_i to WW, update dp=max(dp,dp[wwi]+pi)dp = \max(dp, dp[w - w_i] + p_i). This inner loop iterates over possible multiples implicitly by reusing updated values, yielding a time complexity of O(nW)O(nW) and space O(W)O(W). The approach fills the DP table by considering items in any order, as repetitions allow revisiting subproblems freely. In certain contexts, the UKP relates to the coin change problem, where finding the minimum number of coins (each with denomination wiw_i) to sum exactly to an amount WW equates to an unbounded knapsack minimization variant with unit profits pi=1p_i = 1 for all items, solved via a similar DP recurrence minimizing the count. A greedy strategy can apply under specific conditions on profit-to-weight ratios pi/wip_i / w_i, such as when items are sorted in non-increasing order of these ratios and selecting maximal quantities of the highest-ratio item followed by the next yields optimality; introduced such a greedy method as an approximation that excels in these cases. For illustration, consider two items: one with weight 2 and profit 3, another with weight 3 and profit 4, and capacity W=5W = 5. The DP computes dp{{grok:render&&&type=render_inline_citation&&&citation_id=5&&&citation_type=wikipedia}} = 7 by selecting one of each item (total weight 5), outperforming alternatives like two of the first item (weight 4, profit 6). This example highlights how repetition enables combinations not possible in restricted variants like the 0/1 knapsack.

Bounded Knapsack Problem

The bounded knapsack problem (BKP) is a generalization of the classical in which multiple copies of each item type are available, but the number of copies for each type ii is limited by a positive bound bi1b_i \geq 1. The problem is formulated as follows: given nn item types with weights wi>0w_i > 0, profits pi>0p_i > 0, and bounds bib_i, along with a knapsack capacity W>0W > 0, find nonnegative s xix_i satisfying 0xibi0 \leq x_i \leq b_i for i=1,,ni = 1, \dots, n that maximize the total profit i=1npixi\sum_{i=1}^n p_i x_i subject to the capacity constraint i=1nwixiW\sum_{i=1}^n w_i x_i \leq W. This formulation bridges the 0/1 knapsack problem, where each bi=1b_i = 1, and the unbounded knapsack problem, which corresponds to the limit case as bib_i \to \infty for all ii. Like other knapsack variants, the BKP is NP-hard, but it admits dynamic programming solutions. A standard dynamic programming approach extends the 0/1 knapsack recurrence by treating each of the bib_i copies of item ii as a distinct binary item, leading to a table of size proportional to the total number of copies bi\sum b_i. Alternatively, a modified recurrence can iterate over possible multiples up to bib_i for each item, yielding a of O(nWi=1nbi)O(n W \sum_{i=1}^n b_i). This complexity can be reduced through optimizations, such as binary decomposition of bounds or expanding-core techniques that dynamically adjust the problem core during computation. For illustration, consider a BKP instance with knapsack capacity W=10W = 10 and two item types: type 1 with w1=3w_1 = 3, p1=4p_1 = 4, b1=3b_1 = 3; type 2 with w2=4w_2 = 4, p2=5p_2 = 5, b2=2b_2 = 2. An optimal solution selects two copies of type 1 and one copy of type 2 (x1=2x_1 = 2, x2=1x_2 = 1, total weight 10, profit 13).

Direct Generalizations

Multidimensional Knapsack Problem

The multidimensional knapsack problem (MKP) generalizes the classical 0/1 by incorporating multiple constraints, where each item consumes resources across several dimensions such as weight, volume, or cost, while aiming to maximize total profit without exceeding any capacity limit. This extension arises naturally in scenarios requiring balanced allocation across diverse resources, increasing the problem's computational challenge compared to the single-constraint case. The MKP can be formally defined as the integer linear program that maximizes the objective function i=1npixi\sum_{i=1}^n p_i x_i, subject to the constraints i=1nwi,jxiWj\sum_{i=1}^n w_{i,j} x_i \leq W_j for each j=1,,dj = 1, \dots, d, and xi{0,1}x_i \in \{0, 1\} for all items i=1,,ni = 1, \dots, n, where pip_i is the profit of item ii, wi,jw_{i,j} is its in jj, and WjW_j is the capacity in that . When d=1d=1, the MKP reduces to the standard 0/1 . The MKP is strongly NP-hard, even for d=2d=2, and admits no fully polynomial-time approximation scheme (FPTAS) unless P=NPP = NP, though a polynomial-time approximation scheme (PTAS) exists when the number of dimensions dd is fixed. Exact solution methods include , which dualizes all but one constraint to obtain strong lower bounds via subgradient optimization, often combined with branch-and-bound for optimality. Branch-and-cut approaches, leveraging integer solvers with cutting planes to tighten relaxations, have also proven effective for moderate-sized instances. Applications of the MKP span multi-resource allocation problems, such as scheduling tasks on multiprocessor systems where items represent jobs with constraints on and memory usage, or under multiple risk and liquidity limits. For instance, consider a cargo loading scenario with two dimensions—weight and —where three items have profits p=[60,100,120]p = [60, 100, 120], weight consumptions wi,1=[10,20,30]w_{i,1} = [10, 20, 30] and volume consumptions wi,2=[5,12,10]w_{i,2} = [5, 12, 10], with capacities W1=50W_1 = 50 (weight) and W2=30W_2 = 30 (). The optimal selection (items 1 and 2) yields profit 160, with total weight 30 ≤ 50 and volume 17 ≤ 30, balancing both constraints.

Multiple-Choice Knapsack Problem

The multiple-choice knapsack problem (MCKP) is a generalization of the classical 0/1 knapsack problem in which the set of items is partitioned into disjoint groups or classes, and exactly one item must be selected from each group to maximize the total profit subject to a single capacity constraint. Formally, given rr groups G1,G2,,GrG_1, G_2, \dots, G_r, where group GkG_k contains nkn_k items each with profit pkjp_{k j} and weight wkjw_{k j} for j=1,,nkj = 1, \dots, n_k, and a knapsack capacity WW, the objective is to choose binary decision variables xkj{0,1}x_{k j} \in \{0, 1\} such that j=1nkxkj=1\sum_{j=1}^{n_k} x_{k j} = 1 for each group kk, k=1rj=1nkwkjxkjW\sum_{k=1}^r \sum_{j=1}^{n_k} w_{k j} x_{k j} \leq W, and the total profit k=1rj=1nkpkjxkj\sum_{k=1}^r \sum_{j=1}^{n_k} p_{k j} x_{k j} is maximized. This formulation enforces mutual exclusivity within groups, distinguishing it from the standard 0/1 knapsack where items are selected independently without grouping. The MCKP is NP-hard, as it subsumes the 0/1 knapsack problem (by adding groups with a single dummy item of zero weight and profit alongside real items). Despite its hardness, it admits a dynamic programming (DP) solution due to the integer nature of weights and capacity. The standard DP approach defines fk(j)f_k(j) as the maximum profit achievable using the first kk groups and exactly capacity jj, for k=0,,rk = 0, \dots, r and j=0,,Wj = 0, \dots, W. The recurrence is: fk(j)=max{fk1(j)(if selecting a dummy item of weight 0, profit 0)max1mnk,wkmj(fk1(jwkm)+pkm)f_k(j) = \max \begin{cases} f_{k-1}(j) & \text{(if selecting a dummy item of weight 0, profit 0)} \\ \max_{1 \leq m \leq n_k, \, w_{k m} \leq j} \left( f_{k-1}(j - w_{k m}) + p_{k m} \right) \end{cases} with base cases f0(0)=0f_0(0) = 0 and f0(j)=f_0(j) = -\infty for j>0j > 0 (or handled via initialization). To enforce exactly one selection, the algorithm iterates over groups sequentially, updating the DP table by considering each possible item in the current group and taking the maximum over feasible choices; the time complexity is O(rWnˉ)O(r W \bar{n}), where nˉ\bar{n} is the average number of items per group, making it efficient for moderate WW. Seminal work by Pisinger (1995) introduced a minimal DP algorithm that optimizes space and time by partitioning the problem and using linear-time bounding, achieving practical performance on large instances. Applications of the MCKP arise in scenarios requiring categorized selection under resource limits, such as investment portfolio optimization where assets are grouped by sector (e.g., , healthcare) and exactly one option per sector is chosen to maximize return within a . Other domains include for loading with type-specific choices and for in networks. For instance, consider a simplified portfolio example with three groups and capacity W=12W = 12: Group 1 (: A with profit 10, weight 5; B with 8, 4), Group 2 (healthcare: C with 12, 6; D with 9, 3), Group 3 (: E with 15, 7; F with 11, 5). The optimal selection is B from Group 1, D from Group 2, and F from Group 3, yielding total profit 8 + 9 + 11 = 28 and weight 4 + 3 + 5 = 12. This illustrates group-wise maximization via DP.

Stochastic Knapsack Problem

The stochastic knapsack problem extends the classical by incorporating , where item weights or profits are modeled as random variables drawn from known probability distributions. This introduces probabilistic constraints or objectives, reflecting real-world scenarios such as under demand variability or inventory management with uncertain sizes. The problem was first explored in the chance-constrained setting by Ross and Tsang in 1989, focusing on maximizing deterministic profits while ensuring the probability of capacity violation remains below a threshold. In the one-stage , the objective is to select a of items to maximize the total profit i=1npixi\sum_{i=1}^n p_i x_i, subject to the chance constraint P(i=1nwi(ω)xiW)αP\left(\sum_{i=1}^n w_i(\omega) x_i \leq W\right) \geq \alpha, where xi{0,1}x_i \in \{0,1\} are binary decision variables, pip_i are fixed profits, wi(ω)w_i(\omega) are random weights depending on ω\omega, WW is the knapsack capacity, and α(0,1)\alpha \in (0,1) is the reliability level. For specific distributions like normal, the constraint can be reformulated deterministically using the inverse : i=1nμwixi+Φ1(α)i=1nσwi2xiW\sum_{i=1}^n \mu_{w_i} x_i + \Phi^{-1}(\alpha) \sqrt{\sum_{i=1}^n \sigma_{w_i}^2 x_i} \leq W
Add your contribution
Related Hubs
User Avatar
No comments yet.