Duplicates are the quiet tax you pay in everyday JavaScript. You fetch records from an API, merge results from two services, or combine user inputs from multiple steps—and suddenly your array has repeated entries that break UI lists, inflate analytics, or slow down processing. I see this constantly in production code, and it’s one of those problems that looks trivial until you hit edge cases like objects, NaN, mixed types, or data sets that are big enough to feel slow. If you’ve ever stared at a list of tags that shows the same label three times, you already know why this matters.
I’m going to walk you through the most practical ways to get unique values from a JavaScript array, and I’ll do it the way I’d teach a teammate: clear, honest about trade-offs, and grounded in real scenarios. You’ll learn the standard Set-based approach, loop-based alternatives, and filtering techniques. I’ll also show you when each method is a good fit, what to avoid, and how to deal with tricky data like objects or case-insensitive strings. If you’re building modern apps in 2026, these patterns still show up in everyday work.
The shape of the problem
When I say “unique values,” I mean an array that contains one copy of each distinct value according to some equality rule. For primitives, that rule is usually SameValueZero (the equality used by Set and includes), which means NaN is treated as equal to NaN and +0 equals -0. For objects, “unique” usually means unique by reference unless you supply your own rule.
Here’s what makes this topic harder than it first looks:
- Your data might be primitives, objects, or a mix of both.
- Equality rules differ between methods (indexOf vs includes vs Set).
- Performance can swing from “instant” to “painful” depending on array size.
- You often need stable ordering (preserve the first appearance).
When I pick an approach, I always ask myself three questions:
1) What kind of values am I deduplicating?
2) How large is the array likely to be?
3) Do I need stable order or custom equality?
Those three questions determine the right technique more than any blog snippet.
The Set approach: my default in modern code
If I’m dealing with a flat array of primitive values (numbers, strings, booleans), I reach for Set. It is concise, readable, and typically the fastest for large arrays because it has near O(1) lookup time.
// Example: product IDs from a merged payload
const productIds = [101, 102, 101, 103, 102, 104, 104];
// Unique using Set
const uniqueProductIds = […new Set(productIds)];
console.log(uniqueProductIds); // [101, 102, 103, 104]
If you’re in a functional style, I also like the explicit form with Array.from:
const uniqueProductIds = Array.from(new Set(productIds));
Why I use it:
- Reads clearly.
- Handles NaN correctly (indexOf does not).
- Scales well for large arrays.
The biggest limitation is that it uses reference equality for objects. That’s often fine for arrays of IDs or strings, but it’s not enough for arrays of objects.
How Set handles edge cases
Here’s a quick reality check on how Set behaves with tricky values:
const values = [NaN, NaN, 0, -0, "0", 0, undefined, undefined, null, null];
const unique = […new Set(values)];
console.log(unique);
// [NaN, 0, "0", undefined, null]
Notice that NaN collapses to a single entry, and 0 and -0 are considered the same. That’s usually what you want.
If you want unique values while keeping the first appearance order, Set does that too. It preserves insertion order by design.
The loop with includes: explicit and easy to extend
I still use a loop with includes when I need clarity or custom logic that I’ll likely extend later. It’s not the fastest for large arrays, but it’s readable and easy to debug.
// Example: collecting unique status values in a UI list
const statuses = ["open", "in-progress", "open", "closed", "closed"];
const uniqueStatuses = [];
for (const status of statuses) {
if (!uniqueStatuses.includes(status)) {
uniqueStatuses.push(status);
}
}
console.log(uniqueStatuses); // ["open", "in-progress", "closed"]
This works well for small-to-medium arrays and is great when you need to do extra work on first occurrence. For example, you might want to enrich the object or track the index where it first appeared.
Why includes is better than indexOf here
Includes uses SameValueZero, so it treats NaN as equal to NaN. IndexOf does not—it can’t find NaN. That’s a subtle but real bug if you get NaN from a calculation and expect it to deduplicate.
Filter with indexOf: classic and still useful
The filter + indexOf pattern is classic because it’s a one-liner and still readable to most JavaScript developers. It keeps the first instance and filters out the rest.
const categories = ["camera", "lens", "camera", "tripod", "lens"];
const uniqueCategories = categories.filter((value, index, array) => {
return array.indexOf(value) === index;
});
console.log(uniqueCategories); // ["camera", "lens", "tripod"]
This is compact, but I avoid it for big arrays because it can degrade to O(n^2) time complexity. On arrays with tens of thousands of entries, the Set approach is safer. Also note it mishandles NaN for the same reason as indexOf.
If you prefer filter but want NaN handled correctly, you can switch to findIndex with Object.is, or use includes-based logic (though you still pay the O(n^2) cost).
Modern versus traditional: quick decision table
When I’m coaching a team, I like to summarize the choice this way:
Traditional approach
My recommendation
—
—
filter + indexOf
Use Set for speed and clarity
for loop
Use loop; it’s easy to extend
filter + indexOf
Set is simplest and stable
loop + includes
Use Set or includes-based loop
manual loop
Use Map keyed by propertyNotice how Set wins in most cases with primitives. For object arrays, you usually need a custom key, which I’ll cover next.
Unique values from arrays of objects
This is where most real-world deduplication happens. You don’t want unique object references—you want unique by some key like id, email, or slug.
Unique by id using Map
Map lets you keep the first object per key while preserving order:
const users = [
{ id: "u100", name: "Ava" },
{ id: "u101", name: "Leo" },
{ id: "u100", name: "Ava Martins" },
{ id: "u102", name: "Mina" }
];
const uniqueById = Array.from(
new Map(users.map(user => [user.id, user])).values()
);
console.log(uniqueById);
// [
// { id: "u100", name: "Ava Martins" },
// { id: "u101", name: "Leo" },
// { id: "u102", name: "Mina" }
// ]
This example keeps the last instance for each id because later entries overwrite earlier ones in the Map. If you want the first instance, you can flip the logic.
Keep the first occurrence with a Set of keys
const users = [
{ id: "u100", name: "Ava" },
{ id: "u101", name: "Leo" },
{ id: "u100", name: "Ava Martins" },
{ id: "u102", name: "Mina" }
];
const seen = new Set();
const uniqueById = [];
for (const user of users) {
if (!seen.has(user.id)) {
seen.add(user.id);
uniqueById.push(user);
}
}
console.log(uniqueById);
// [
// { id: "u100", name: "Ava" },
// { id: "u101", name: "Leo" },
// { id: "u102", name: "Mina" }
// ]
This is my go-to when the first record wins, which is common in logs, event streams, or audit data.
Case-insensitive uniqueness for strings
For tags, labels, or emails, you often want uniqueness ignoring case:
const tags = ["JavaScript", "Node", "node", "API", "api", "Api"];
const seen = new Set();
const uniqueTags = [];
for (const tag of tags) {
const key = tag.toLowerCase();
if (!seen.has(key)) {
seen.add(key);
uniqueTags.push(tag);
}
}
console.log(uniqueTags); // ["JavaScript", "Node", "API"]
This keeps the original casing for the first occurrence but enforces case-insensitive uniqueness. That pattern is extremely common in real products.
Performance considerations that matter in practice
I’m not going to pretend you always need to micro-tune performance, but you should understand the shape of the cost.
- Set and Map give you near constant-time lookups, so overall time is usually linear. For large arrays, this makes a big difference.
- Loop + includes and filter + indexOf can degrade to quadratic time because each lookup scans the array.
- For arrays of a few hundred entries, you probably won’t notice. For arrays of tens of thousands, you will.
If I had to give a rough rule of thumb:
- Under 1,000 items: any approach is fine.
- 1,000 to 50,000 items: use Set or Map.
- Over 50,000 items: also watch memory and consider streaming or chunking.
In modern front-end apps, I also think about render cycles. If you deduplicate on every keystroke, even 10-15ms can feel like lag. That’s why I often do deduping once on load, or memoize it.
Common mistakes I see in code reviews
I’ll call these out because they’re easy to miss.
1) Using indexOf for NaN
const values = [1, NaN, 2, NaN];
const unique = values.filter((v, i, arr) => arr.indexOf(v) === i);
console.log(unique); // [1, NaN, 2, NaN] <- wrong
NaN never equals itself using strict equality, so indexOf can’t find it. If NaN is possible, use Set or includes-based logic.
2) Deduplicating objects by reference
const items = [{ id: 1 }, { id: 1 }];
const unique = […new Set(items)];
console.log(unique.length); // 2
Set doesn’t know that these are “the same” by id. If you want uniqueness by field, you need a custom key.
3) Forgetting to preserve order when needed
Some methods keep order, some don’t. Set preserves insertion order, Map does too. Sorting then deduplicating can change the order and break UI expectations. I only sort if the product requirement says ordering doesn’t matter.
4) Deduplicating by JSON.stringify without thinking
I sometimes see this:
const unique = Array.from(
new Map(items.map(item => [JSON.stringify(item), item])).values()
);
This can work but is fragile: property order matters, circular references break it, and it can be slow. I only use this if I control the data shape and know it’s safe.
Picking the right method: a practical guide
Here’s how I make the call quickly:
- If values are primitives: use new Set().
- If values are objects: use Map or Set with a key property.
- If you need a custom equality rule: use a loop and a Set of normalized keys.
- If the array is small and readability is the top priority: use loop + includes.
And yes, it’s okay to favor readability over micro-optimization for small data sets. Just be aware of where the line is.
Real-world scenarios
Scenario 1: Unique tags from user content
You’re building a tag cloud from posts. Tags are messy and have different casing.
const posts = [
{ title: "API Design", tags: ["API", "Design", "Node"] },
{ title: "Node Performance", tags: ["node", "Performance"] },
{ title: "JS Patterns", tags: ["JavaScript", "design"] }
];
const allTags = posts.flatMap(post => post.tags);
const seen = new Set();
const uniqueTags = [];
for (const tag of allTags) {
const key = tag.toLowerCase();
if (!seen.has(key)) {
seen.add(key);
uniqueTags.push(tag);
}
}
console.log(uniqueTags);
// ["API", "Design", "Node", "Performance", "JavaScript"]
That gives you a clean list without losing the original casing of the first occurrence.
Scenario 2: Unique products by SKU
const cartItems = [
{ sku: "A12", name: "Tripod", quantity: 1 },
{ sku: "B55", name: "Microphone", quantity: 1 },
{ sku: "A12", name: "Tripod", quantity: 2 }
];
const uniqueBySku = Array.from(
new Map(cartItems.map(item => [item.sku, item])).values()
);
console.log(uniqueBySku);
// Keeps the last entry for each SKU
If you need to combine quantities instead of picking one, that’s a different problem. But the Map pattern is still the right starting point.
Scenario 3: Unique numeric telemetry values with NaN
const readings = [12.1, NaN, 12.1, 13.8, NaN];
const uniqueReadings = […new Set(readings)];
console.log(uniqueReadings); // [12.1, NaN, 13.8]
This is a small example, but if your telemetry arrays are big, Set is also the fastest for this case.
When not to remove duplicates
Sometimes, duplicates are part of the signal. Here’s when I avoid deduplicating:
- When you’re counting frequency (duplicates are data).
- When you need to preserve event order for debugging.
- When duplicates represent different entities with identical values.
I’ve seen teams “clean” data and then lose the ability to detect anomalies. Be sure you’re removing duplicates for a reason, not because it feels like a cleanup step.
A deeper look at equality: SameValueZero
Most of these techniques rely on what JavaScript considers equal. Set and includes use SameValueZero, which is similar to strict equality but treats NaN as equal to itself.
If you need a custom equality (for example, numbers rounded to 2 decimals), you should normalize first:
const prices = [9.995, 10.004, 10.0, 10.01];
const seen = new Set();
const uniquePrices = [];
for (const price of prices) {
const rounded = Math.round(price * 100) / 100;
if (!seen.has(rounded)) {
seen.add(rounded);
uniquePrices.push(rounded);
}
}
console.log(uniquePrices); // [10.0, 10.01]
This turns the problem into a normalization step plus a Set, which is an easy mental model to carry between projects.
Deduping in a pipeline: filter + Set
If you prefer a pipeline style, you can combine normalization and Set in a more declarative way. I use this when the steps are short and the code remains readable.
const emails = [
];
const uniqueEmails = Array.from(
new Set(emails.map(email => email.toLowerCase()))
);
console.log(uniqueEmails);
// ["[email protected]", "[email protected]", "[email protected]"]
The downside is that you lose original casing. If you care about the original value, use a Set of normalized keys and push the first original, like I showed earlier.
Memory trade-offs
Set and Map use extra memory proportional to the number of unique values. That’s almost always acceptable, but if you’re working in a constrained environment or with huge streams, you might need to deduplicate incrementally rather than holding everything in memory at once.
In front-end apps, the memory overhead is usually fine because you’re deduping a list that you already have in memory. But on backend services, it’s easy to accidentally load millions of rows into RAM and then create a Set on top of it. When I’m dealing with large volumes, I use chunked processing (pull 10k rows, dedupe, store a rolling Set of keys) or I let the database handle uniqueness upstream.
Stable order vs sorted order
One reason Set and Map are so nice is that they preserve insertion order. That means the “first time I saw this value” is what you get in the output.
Sometimes, though, you want a sorted list of unique values. A common pattern is:
const values = [3, 1, 2, 3, 2, 4, 1];
const uniqueSorted = […new Set(values)].sort((a, b) => a – b);
This is fine, but notice it does two passes: one to dedupe and one to sort. If the requirement is “sorted unique values” and you don’t care about original order, you can sort first and then remove duplicates in one pass, which can be cheaper for very large arrays:
const values = [3, 1, 2, 3, 2, 4, 1];
const sorted = values.slice().sort((a, b) => a – b);
const uniqueSorted = [];
for (const v of sorted) {
if (uniqueSorted.length === 0 || uniqueSorted[uniqueSorted.length – 1] !== v) {
uniqueSorted.push(v);
}
}
This approach is still O(n log n) because of sorting, but it does avoid the hash set overhead. I use it when order does not matter and the output should be sorted anyway.
Deduplicating mixed-type arrays
Mixed types are common when data comes from loose sources (forms, query params, CSVs). For example:
const values = ["1", 1, "01", 1, true, "true", false, 0];
With Set, each distinct value by SameValueZero is unique, so you’ll keep both "1" and 1, "true" and true, and so on. That’s correct for most logic, but if you actually want to treat "1" and 1 as the same, you need to normalize to a common representation:
const values = ["1", 1, "01", 1, true, "true", false, 0];
const seen = new Set();
const unique = [];
for (const value of values) {
const key = String(value).toLowerCase();
if (!seen.has(key)) {
seen.add(key);
unique.push(value);
}
}
Now you’ll keep only one value per string representation. This is useful for loose data, but it’s dangerous if you need strict typing. The key takeaway is: always decide which equality you really want, then build a key for it.
Unique values with a reusable helper
If your project deduplicates in multiple places, I recommend a small helper that makes the intent obvious. I keep it simple and avoid over-engineering:
const uniqueBy = (items, keyFn) => {
const seen = new Set();
const result = [];
for (const item of items) {
const key = keyFn(item);
if (!seen.has(key)) {
seen.add(key);
result.push(item);
}
}
return result;
};
Usage examples:
const uniqueEmails = uniqueBy(users, user => user.email.toLowerCase());
const uniqueSkus = uniqueBy(cartItems, item => item.sku);
I like this pattern because it forces you to define the equality rule in one line. It also keeps the loop fast and predictable.
Using reduce for deduplication
Reduce is not the fastest or cleanest for this, but some teams like it for functional pipelines. If you do use it, pair it with a Set to keep it efficient:
const values = ["a", "b", "a", "c", "b"];
const unique = values.reduce((acc, value) => {
if (!acc.seen.has(value)) {
acc.seen.add(value);
acc.items.push(value);
}
return acc;
}, { seen: new Set(), items: [] }).items;
It’s more verbose than a loop, but it stays linear. I avoid reduce if the readability suffers, but it’s a valid style if the codebase already leans functional.
Objects: unique by multiple keys
Sometimes, one field is not enough. Example: you want unique by both country and city.
const locations = [
{ country: "US", city: "Austin" },
{ country: "US", city: "Austin" },
{ country: "US", city: "Denver" },
{ country: "CA", city: "Toronto" }
];
const uniqueLocations = uniqueBy(
locations,
loc => ${loc.country}|${loc.city}
);
This composite key trick works well for small keys. If you’re worried about a separator conflict, you can use JSON.stringify on the key tuple:
const uniqueLocations = uniqueBy(
locations,
loc => JSON.stringify([loc.country, loc.city])
);
I still keep it simple when I can. Composite string keys are usually good enough.
Deep equality: when objects are truly complex
If you need deep equality (same shape and values) across objects, you have two main options:
1) Generate a stable hash/key for each object.
2) Use a deep comparison function to check every candidate.
Both can be expensive. In practice, I avoid deep equality for large lists unless it’s truly required. A stable key approach can work if you control the data shape and can produce a canonical representation (sorted keys, no cycles). Otherwise, I define a specific subset of fields that define identity. It’s not perfect, but it’s more reliable and faster.
Sparse arrays and “holes”
JavaScript arrays can have empty slots. This is rare, but it shows up when people do new Array(5) or delete indices. Set will treat missing slots as missing; they aren’t values. Includes skips holes as well when iterating with for…of. If your data might be sparse, the safest approach is to explicitly iterate with a classic for loop and check i in array:
const arr = new Array(5);
arr[1] = "a";
arr[3] = "a";
const unique = [];
const seen = new Set();
for (let i = 0; i < arr.length; i++) {
if (!(i in arr)) continue;
const value = arr[i];
if (!seen.has(value)) {
seen.add(value);
unique.push(value);
}
}
Most teams never need this, but it’s good to know the edge case exists.
BigInt, Symbol, and other special values
Set handles BigInt and Symbol just fine, but it deduplicates by identity for Symbol. Two Symbols with the same description are still unique:
const values = [Symbol("x"), Symbol("x")];
const unique = […new Set(values)];
console.log(unique.length); // 2
If your array contains Symbols and you want to group by description, you need a custom key. With BigInt, Set behaves as expected: 1n is not equal to 1, so mixed numeric types remain separate.
Streaming and chunked deduplication
When I’m working with huge inputs, I avoid building giant arrays in memory just to dedupe them. Here’s a pattern I’ve used in Node services where data is processed in chunks:
const seen = new Set();
const output = [];
const handleChunk = chunk => {
for (const item of chunk) {
if (!seen.has(item.id)) {
seen.add(item.id);
output.push(item);
}
}
};
// Imagine chunks coming from a stream or paginated API
handleChunk(page1);
handleChunk(page2);
handleChunk(page3);
This keeps memory predictable and avoids repeated work. It also makes it easier to integrate with streaming systems where you may never have the full dataset at once.
Deduping in UI state: memoization and reactivity
In front-end frameworks, the performance hit often comes from repeating the same deduping on every render. I usually memoize the result or compute it once when input changes:
// Pseudocode, framework-agnostic
let cachedInput = null;
let cachedOutput = null;
const uniqueOnce = input => {
if (input === cachedInput) return cachedOutput;
cachedInput = input;
cachedOutput = […new Set(input)];
return cachedOutput;
};
This isn’t about shaving microseconds; it’s about avoiding repeated work in render loops or reactive chains.
Comparing approaches with practical trade-offs
Here’s a richer, real-world comparison to use when you’re deciding in a code review:
Handles NaN
Works for objects
Best use case
—
—
—
Yes
Reference only
Large primitive arrays
Yes
Reference only
Small arrays, custom logic
No
Reference only
Simple primitives, tiny arrays
Yes
Yes
Objects with unique key
Yes
Primitives
Need sorted outputI’m a big fan of Set or Map for production code, mostly because it makes performance and edge cases predictable.
Common pitfalls with object keys
A few more mistakes I see:
- Using mutable keys: if
user.idcan change later, your Set/Map will not update. It’s better to dedupe based on immutable identifiers. - Using undefined keys: if some objects are missing the key, you may treat them all as the same (because key is undefined). Guard against it:
const uniqueById = uniqueBy(items, item => item.id ?? missing:${item.tmpKey});
- Mixing types for the key: if you use numbers and strings as keys, Map treats them as different. Normalize if needed.
Deduplicating while merging data
Often you want to dedupe and merge fields instead of picking first or last. For example, merging counts:
const cartItems = [
{ sku: "A12", name: "Tripod", quantity: 1 },
{ sku: "B55", name: "Microphone", quantity: 1 },
{ sku: "A12", name: "Tripod", quantity: 2 }
];
const bySku = new Map();
for (const item of cartItems) {
if (!bySku.has(item.sku)) {
bySku.set(item.sku, { …item });
} else {
const existing = bySku.get(item.sku);
existing.quantity += item.quantity;
}
}
const merged = Array.from(bySku.values());
This is a common pattern in carts, analytics, and event aggregation. I like it because the Map owns the uniqueness, and the merge logic stays local.
How to test deduping logic
I rarely trust deduping logic without a few tests, because edge cases are easy to miss. If I’m writing tests, I focus on:
- Primitives with duplicates (basic case)
- NaN present
- Mixed types ("1" vs 1)
- Objects with same key
- Order preservation
A quick test outline looks like this:
const input = ["a", "b", "a", "c"];
expect(unique(input)).toEqual(["a", "b", "c"]);
const nanInput = [NaN, 1, NaN];
expect(unique(nanInput)).toEqual([NaN, 1]);
const objects = [{ id: 1 }, { id: 1 }, { id: 2 }];
expect(uniqueBy(objects, o => o.id).length).toBe(2);
If you only remember one thing: test NaN and object keys. Those are the most common hidden bugs.
Alternatives and libraries
If you’re using a utility library like Lodash, there are helpers like uniq and uniqBy. I still prefer native Set and a small helper for most cases, because it reduces dependency surface and makes the equality rule explicit. But in a codebase that already uses Lodash, uniqBy is a decent choice for object arrays.
Production considerations
Deduping is rarely the final step in a pipeline. It usually sits between fetching and rendering, or between ingestion and storage. That means you should think about where the dedupe should live:
- In the database layer (unique index, DISTINCT) when possible.
- At the service layer if you need custom rules or more context.
- At the UI layer only when the data is small or already in memory.
If the data can be filtered at the source (like SQL DISTINCT or API dedupe parameters), it’s often cheaper to do it there. I still use JavaScript deduping frequently, but I don’t want it to compensate for missing upstream constraints.
A practical checklist
When I implement deduping, I run through this quick list:
- What counts as “same”? (strict, case-insensitive, rounded, by id)
- Do I need to keep the first or last occurrence?
- Is order important?
- How big can the array get?
- Is there an upstream place that should enforce uniqueness?
This checklist saves me from choosing the wrong method and having to rewrite later.
Summary you can reuse
If you want the short version to remember:
- Use
new Set()for primitive arrays. It’s fast, readable, and handles NaN. - Use
Mapor a Set of keys for object arrays. Decide if first or last wins. - Normalize your values if your equality is custom (case-insensitive, rounded, etc.).
- Avoid indexOf for NaN and avoid JSON.stringify for deep objects unless you control the shape.
- For huge arrays, consider streaming or upstream dedupe.
If you internalize those points, you’ll handle 95% of unique-value problems cleanly. The rest are edge cases—and now you know how to handle those too.
