Removing Duplicate Lines Based on a Specific String: Two Similar Cases
In this article, we will discuss two similar cases where the goal is to remove duplicate lines based on a specific string using different programming languages. The first case will be implemented in Python, while the second case will be in JavaScript. By the end of this article, you will have a solid understanding of how to tackle this problem in different programming environments.
Case 1: Removing Duplicate Lines Based on a Specific String in Python
Suppose we have a text file with the following content:
apples
oranges
apples
pears
pears
grapes
Our goal is to remove duplicate lines based on the string "apples", "pears", and keep only the first occurrence of each. The expected output should be:
apples
oranges
pears
grapes
Here's the Python code to achieve this:
def remove_duplicates(input_file, output_file, specific_string):
with open(input_file, 'r') as f:
lines = f.readlines()
unique_lines = []
current_line = None
for line in lines:
if line.strip() == specific_string:
if current_line is None or current_line.strip() != specific_string:
unique_lines.append(line)
current_line = line
elif current_line is None or current_line.strip() != line.strip():
unique_lines.append(line)
current_line = line
with open(output_file, 'w') as f:
f.writelines(unique_lines)
To use this function, simply call it with the input file, output file, and the specific string as arguments:
remove_duplicates('input.txt', 'output.txt', 'apples')
Case 2: Removing Duplicate Lines Based on a Specific String in JavaScript
Now let's implement the same functionality in JavaScript. Suppose we have the following array:
const fruits = [
'apples',
'oranges',
'apples',
'pears',
'pears',
'grapes'
]
Our goal is to remove duplicate entries based on the string "apples", "pears", and keep only the first occurrence of each. The expected output should be:
const fruits = [
'apples',
'oranges',
'pears',
'grapes'
]
Here's the JavaScript code to achieve this:
function removeDuplicates(array, specificString) {
let uniqueArray = [];
let currentElement = null;
for (let element of array) {
if (element === specificString) {
if (currentElement === null || currentElement !== specificString) {
uniqueArray.push(element);
currentElement = element;
}
} else if (currentElement === null || currentElement !== element) {
uniqueArray.push(element);
currentElement = element;
}
}
return uniqueArray;
}
To use this function, simply call it with the array and the specific string as arguments:
const fruits = [
'apples',
'oranges',
'apples',
'pears',
'pears',
'grapes'
];
const uniqueFruits = removeDuplicates(fruits, 'apples');
- In this article, we discussed two similar cases where the goal is to remove duplicate lines based on a specific string.
- The first case was implemented in Python, where we read a text file, removed duplicate lines based on a specific string, and wrote the result to an output file.
- The second case was implemented in JavaScript, where we removed duplicate entries based on a specific string from an array.
References
-
Type: Article
Title: Removing duplicate lines from a file in Python
URL: https://stackoverflow.com/questions/1510533/removing-duplicate-lines-from-a-file-in-python
-
Type: Article
Title: Remove duplicates from an array and object in JavaScript
URL: https://gomakethings.com/remove-duplicates-from-an-array-and-object-in-javascript/