112

I've got this code:

numbers = list(range(1, 50))

for i in numbers:
    if i < 20:
        numbers.remove(i)

print(numbers)

This code should go through all the elements of the list and remove those elements that are less than 20. But, the result I'm getting is:

[2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49]

As you can see, the code removed all even numbers less than 20, but left the odd ones. Looks like I'm doing something wrong with the remove.

Improve this question
2
  • See also: stackoverflow.com/questions/1207406/…. I reconsidered and decided that this is not a duplicate; this question is about understanding the failure of one specific wrong way to approach the problem, while the other question is about finding correct ways. Commented Jul 30, 2022 at 0:29
  • See also: stackoverflow.com/questions/30283909/… for the case where insertion inside a for loop causes the for loop to not terminate (not stopping). Commented Feb 9, 2024 at 4:10

12 Answers 12

Reset to default
161

You're modifying the list while you iterate over it. That means that the first time through the loop, i == 1, so 1 is removed from the list. Then the for loop goes to the second item in the list, which is not 2, but 3! Then that's removed from the list, and then the for loop goes on to the third item in the list, which is now 5. And so on. Perhaps it's easier to visualize like so, with a ^ pointing to the value of i:

[1, 2, 3, 4, 5, 6...]
 ^

That's the state of the list initially; then 1 is removed and the loop goes to the second item in the list:

[2, 3, 4, 5, 6...]
    ^
[2, 4, 5, 6...]
       ^

And so on.

There's no good way to alter a list's length while iterating over it. The best you can do is something like this:

numbers = [n for n in numbers if n >= 20]

or this, for in-place alteration (the thing in parens is a generator expression, which is implicitly converted into a tuple before slice-assignment):

numbers[:] = (n for n in numbers if n >= 20)

If you want to perform an operation on n before removing it, one trick you could try is this:

for i, n in enumerate(numbers):
    if n < 20:
        print("do something")
        numbers[i] = None
numbers = [n for n in numbers if n is not None]
Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

Related note on for keeping an index from the Python docs docs.python.org/3.9/reference/…: "There is a subtlety when the sequence is being modified by the loop (this can only occur for mutable sequences, e.g. lists). An internal counter is used to keep track of which item is used next, and this is incremented on each iteration. ... This means that if the suite deletes the current (or a previous) item from the sequence, the next item will be skipped (since it gets the index of the current item which has already been treated)."
This is a good answer, but with the final solution, "if you want to perform an operation...", is slightly unsatisfactory because 1) in fact there is no need to include that qualification: it is just a waste of effort to try and remove elements while iterating in a single operation, so this 2-stage solution applies in all cases, and 2) because there should be a warning that setting to None won't always be appropriate (if some elements are meant to be None): some conventional "poison pill" value, appropriate to the case in hand, is needed.
When I say that about the "waste of effort", I am referring to the "general" solution by the way, i.e. when random elements may need removing, rather than the "specialist" case of removing elements only at the start (or only at the end), which lends itself to something simple, like your list comprehension solution...
29

Begin at the list's end and go backwards:

li = list(range(1, 15))
print(li)

for i in range(len(li) - 1, -1, -1):
    if li[i] < 6:
        del li[i]
        
print(li)

Result:

[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14] 
[6, 7, 8, 9, 10, 11, 12, 13, 14]
Improve this answer

5 Comments

This is a very specialised answer: it is not in fact clear whether we're meant to be looking for a general solution to the problem of how to remove elements while iterating, or how to do this exclusively when we just want to remove the first n elements of a list. The chosen answer provides the former, which is infinitely more helpful, but also the latter, in the shape of a one-line list comprehension solution.
@mikerodent no it's not. It's pretty common when you want to modify a list while iterating over it that going backwards works
@Boris you haven't understood my comment. The OP's question does not specify that we are removing contiguous elements (either from the start or end of the list).
I still don't understand your comment then because it doesn't matter whether the list is shuffled or ordered, this code will still work.
This is where my intuition led me too. The part that you still have to iterate over won't be affected at all!
11

@senderle's answer is the way to go!

Having said that to further illustrate even a bit more your problem, if you think about it, you will always want to remove the index 0 twenty times:

[1,2,3,4,5............50]
 ^
[2,3,4,5............50]
 ^
[3,4,5............50]
 ^

So you could actually go with something like this:

aList = list(range(50))
i = 0
while i < 20:
    aList.pop(0)
    i += 1

print(aList) #[21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49]

I hope it helps.

The ones below are not bad practices AFAIK.

EDIT (Some more):

lis = range(50)
lis = lis[20:]

Will do the job also.

EDIT2 (I'm bored):

functional = filter(lambda x: x> 20, range(50))
Improve this answer

Comments

3

As an additional information to @Senderle's answer, just for records, I thought it's helpful to visualize the logic behind the scene when python sees for on a "Sequence type".

Let's say we have :

lst = [1, 2, 3, 4, 5]

for i in lst:
    print(i ** 2)

It is actually going to be :

index = 0
while True:
    try:
        i = lst.__getitem__(index)
    except IndexError:
        break
    print(i ** 2)
    index += 1

That's what it is, there is a try-catch mechanism that for has when we use it on a Sequence types or Iterables(It's a little different though - calling next() and StopIteration Exception).

*All I'm trying to say is, python will keep track of an independent variable here called index, so no matter what happens to the list (removing or adding), python increments that variable and calls __getitem__() method with "this variable" and asks for item.

Improve this answer

1 Comment

Technically, while recalls the condition every times you do a loop, so you could verify for the index to be out of range in while's condition and not worry about a try-except ? Or maybe I'm wrong, I'm very new to python.
3

Building on and simplying the answer by @eyquem ...

The problem is that elements are being yanked out from under you as you iterate, skipping numbers as you progress to what was the next number.

If you start from the end and go backwards, removing items on-the-go won't matter, because when it steps to the "next" item (actually the prior item), the deletion does not affect the first half of the list.

Simply adding reversed() to your iterator solves the problem. A comment would be good form to preclude future developers from "tidying up" your code and breaking it mysteriously.

for i in reversed(numbers): # `reversed` so removing doesn't foobar iteration
  if i < 20:
    numbers.remove(i)
Improve this answer

1 Comment

That reasoning for your solution with "the deletion does not affect the first half" is only correct if there are no duplicates that get deleted.
3

So I found a solution but it's really clumsy...

First of all you make an index array, where you list all the index' you want to delete like in the following

numbers = range(1, 50)
index_arr = []

for i in range(len(numbers)):
    if numbers[i] < 20:
        index_arr.append(i)

after that you want to delete all the entries from the numbers list with the index saved in the index_arr. The problem you will encounter is the same as before. Therefore you have to subtract 1 from every index in the index_arr after you just removed a number from the numbers arr, like in the following:

numbers = range(1, 50)
index_arr = []

for i in range(len(numbers)):
    if numbers[i] < 20:
        index_arr.append(i)

for del_index in index_list:
    numbers.pop(del_index)

    #the nasty part
    for i in range(len(index_list)):
        index_list[i] -= 1

It will work, but I guess it's not the intended way to do it

Improve this answer

1 Comment

You have a couple of syntax errors in your code, with missing parentheses.
1

Making a shallow copy of numbers for iteration, will ensure that the list use for iteration is not modified. The shallow copy can be made using list(), or copy.copy(), and will ensure that the identify of the elements to remove is the same, and reduce the size of the temporary list. Code:

...
for i in list(numbers):
    if i < 20:
        numbers.remove(i)
...
Improve this answer

Comments

0

You could also use continue to ignore the values less than 20

mylist = []

for i in range(51):
    if i<20:
        continue
    else:
        mylist.append(i)
print(mylist)
Improve this answer

Comments

0

Since Python 3.3 you may use the list copy() method as the iterator:

numbers = list(range(1, 50))

for i in numbers.copy():
    if i < 20:
        numbers.remove(i)
print(numbers)

[20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49]
Improve this answer

1 Comment

This looks like a neat suggestion to a classic problem, but I'm not sure that in practice it will prove better than the 2-stage solution suggested at the end of the chosen answer: firstly, what is the cost of the copy operation? Secondly, you have to concern yourself with whether this is a deep or shallow copy. In this trivial case the question doesn't arise but that won't always be so.
0

You can use list() for numbers to create a different copied numbers as shown below:

numbers = list(range(1, 50))
       # ↓ ↓ Here ↓ ↓
for i in list(numbers):
    if i < 20:
        numbers.remove(i)

print(numbers) # [20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 
               #  31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 
               #  f42, 43, 44, 45, 46, 47, 48, 49]
Improve this answer

Comments

0

Well, I needed a sort of this.
The answer was to use while statement.
It really helped me in my case. So I'd like to post it as an answer.

numbers = list(range(1, 50))
while numbers:
    current_number = numbers[0]
    if current_number < 20:
        numbers.remove(current_number)
    else:
        break
numbers
>>> [20, 21, 22, 23, 24, 25, ..., 49]

I also wanted to add my special case using while and for using [:], but it might be out of the scope, so I'll keep it.

Improve this answer

Comments

0

A bit late but I feel like adding my answer. I found the following trick which is fairly simply

for x in list[::-1]:
    if validate(x):
        list.pop(x)

This way, you iterate starting from the end. When you remove the n-th element, the elements n+1,n+2... decrease their index by one, but because you are going backwards this won't intact the following (or preceding, in this case) elements..

Improve this answer

Comments

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.