Remove Duplicate Lines
Strip duplicate lines from a list while preserving order. Toggle case sensitivity, trim leading and trailing whitespace before comparing, and choose whether to keep the first or last occurrence of each duplicate.
How to use
- 1
Paste your list
Drop one item per line. Works for URLs, emails, IDs, keywords or any list where duplicates should be removed.
- 2
Pick options
Toggle case sensitivity, choose whether to trim whitespace before comparing, and pick whether to keep the first or last occurrence of each duplicate.
- 3
Remove duplicates
Click Remove duplicates. The before/after/removed counters update so you can see how many lines were stripped.
- 4
Copy or save
Use Copy to put the result on your clipboard, or Save to download unique-lines.txt.
Common use cases
- Deduplicate a list of email addresses before importing to a CRM.
- Remove duplicate URLs from a sitemap before submitting to search engines.
- Tidy a list of keywords exported from a research tool.
- Strip duplicate lines from a log file before sharing.
Limitations
- Order is preserved — the output order matches the order of first (or last) occurrence in the input.
- Trim whitespace only affects comparison; original lines are kept verbatim in the output.
- Inputs larger than 1 MB are flagged but still processed on the main thread.
Frequently asked questions
Related tools
View allSort Lines
Sort lines alphabetically, numerically or in reverse.
Remove Empty Lines
Strip blank lines from text instantly.
Find and Replace
Find and replace text with optional regex and case sensitivity.
Text Cleaner
Clean text by removing extra spaces, empty lines and special characters.