Chained Assignment and the Copy-vs-View Trap

Real-World Scenario

A pipeline filters a DataFrame down to the rows it needs to update, assigns a new value to one of its columns, and moves on — a pattern nearly every Pandas user writes without thinking twice. Sometimes it updates the original DataFrame. Sometimes it raises an error. Sometimes it does neither: it silently updates a throwaway copy, and the bug ships to production.

The Loud Failure: One-Line Chained Assignment

This filters the positive rows and assigns to y in a single chained expression.

import warnings
import pandas as pd

df = pd.DataFrame({"x": [1, -2, 3, -4], "y": [10, 20, 30, 40]})

with warnings.catch_warnings(record=True) as caught:
    warnings.simplefilter("always")
    df[df["x"] > 0]["y"] = 999

for w in caught:
    print(f"{w.category.__name__}: {w.message}")
print(df)

$ python chain1.py
ChainedAssignmentError: A value is being set on a copy of a DataFrame or Series through chained assignment.
Such chained assignment never works to update the original DataFrame or Series, because the intermediate object on which we are setting values always behaves as a copy (due to Copy-on-Write).

Try using '.loc[row_indexer, col_indexer] = value' instead, to perform the assignment in a single step.

See the documentation for a more detailed explanation: https://pandas.pydata.org/pandas-docs/stable/user_guide/copy_on_write.html#chained-assignment
   x   y
0  1  10
1 -2  20
2  3  30
3 -4  40

df[df["x"] > 0] evaluates first and returns a brand-new DataFrame — the filtered rows, copied out of df. ["y"] = 999 then assigns onto that temporary object, which is immediately discarded, so df prints unchanged. Pandas 3.0’s Copy-on-Write model detects this exact shape (indexing followed immediately by assignment) and emits a ChainedAssignmentError warning — here captured with warnings.catch_warnings so it prints cleanly; run normally, Python prints it to stderr with the script’s full path and line number. Older Pandas versions surfaced the same situation as a SettingWithCopyWarning. Either way it is a warning, not an exception: the script keeps running with the wrong data.

The Silent Failure: Two-Step Chained Assignment

Here the filtered rows are stored in a variable first, then assigned to.

import pandas as pd

df = pd.DataFrame({"x": [1, -2, 3, -4], "y": [10, 20, 30, 40]})

positive = df[df["x"] > 0]
positive["y"] = 999

print("df (original):")
print(df)
print("\npositive (the filtered copy):")
print(positive)

$ python chain2.py
df (original):
   x   y
0  1  10
1 -2  20
2  3  30
3 -4  40

positive (the filtered copy):
   x    y
0  1  999
2  3  999

No error. No warning. positive is updated; df is not — and there is nothing in the output to tell you that. Storing the filtered result in a variable first breaks the pattern Pandas checks for, so Copy-on-Write can no longer tell this apart from an intentional, independent copy. This is the genuinely dangerous version of the bug: the code runs clean, the tests that only check positive pass, and df quietly never changed.

The Fix: One .loc[] Call

The same update, with the row filter and the column passed to one .loc[] call.

import pandas as pd

df = pd.DataFrame({"x": [1, -2, 3, -4], "y": [10, 20, 30, 40]})

df.loc[df["x"] > 0, "y"] = 999

print(df)

$ python chain3.py
   x    y
0  1  999
1 -2   20
2  3  999
3 -4   40

.loc[row_indexer, col_indexer] = value passes both the row filter and the column name into a single call. Pandas resolves the whole assignment against df directly — there’s no intermediate object for the write to land on instead. This is the fix for both failure modes above.

Practice

1. Reproduce the two-step silent failure from this lesson, then add positive = positive.copy() right after the filter, before assigning to positive["y"]. Does the warning behavior change? Does df get updated? Explain why calling .copy() doesn’t fix the underlying bug.

   Solution

   import pandas as pd
   
   df = pd.DataFrame({"x": [1, -2, 3, -4], "y": [10, 20, 30, 40]})
   positive = df[df["x"] > 0].copy()
   positive["y"] = 999
   print(df)
   print(positive)

   $ python practice_chain1.py
      x   y
   0  1  10
   1 -2  20
   2  3  30
   3 -4  40
      x    y
   0  1  999
   2  3  999

   Still no warning, and df is still unchanged — .copy() makes the intent explicit (you’re now deliberately working on an independent copy) but doesn’t change the fact that positive was never connected to df in the first place. The bug isn’t the missing warning; it’s writing to the wrong object. Only .loc[] on df itself fixes that.

2. Write a version of the fix that updates y to 0 for every row where x is negative, using .loc[].

   Solution

   import pandas as pd
   
   df = pd.DataFrame({"x": [1, -2, 3, -4], "y": [10, 20, 30, 40]})
   df.loc[df["x"] < 0, "y"] = 0
   print(df)

   $ python practice_chain2.py
      x   y
   0  1  10
   1 -2   0
   2  3  30
   3 -4   0

3. df["y"] = 999 (no row filter, just a plain column assignment) never triggers this problem. Why not — what’s different about assigning to a whole column versus a filtered slice of one?

   Solution

   import warnings
   import pandas as pd
   
   df = pd.DataFrame({"x": [1, -2, 3, -4], "y": [10, 20, 30, 40]})
   with warnings.catch_warnings(record=True) as caught:
       warnings.simplefilter("always")
       df["y"] = 999
   print("warnings:", len(caught))
   print(df)

   $ python practice_chain3.py
   warnings: 0
      x    y
   0  1  999
   1 -2  999
   2  3  999
   3 -4  999

   df["y"] = 999 is a single indexing operation directly on df, not a chain of two. There’s no intermediate filtered object involved at all — Pandas assigns straight to the column on the original DataFrame, which is exactly what .loc[] also does. The problem only appears once a row filter is chained together with a column assignment.

4. Predict what df.loc[df["x"] > 0]["y"] = 999 does — .loc[] used for the filter, but then a second ["y"] chained after it instead of passing both into one call. Run it and check your prediction.

   Solution

   import warnings
   import pandas as pd
   
   df = pd.DataFrame({"x": [1, -2, 3, -4], "y": [10, 20, 30, 40]})
   with warnings.catch_warnings(record=True) as caught:
       warnings.simplefilter("always")
       df.loc[df["x"] > 0]["y"] = 999
   print([w.category.__name__ for w in caught])
   print(df)

   $ python practice_chain4.py
   ['ChainedAssignmentError']
      x   y
   0  1  10
   1 -2  20
   2  3  30
   3 -4  40

   It fails exactly like the plain df[df["x"] > 0]["y"] = 999 case — ChainedAssignmentError, df unchanged. Using .loc[] for the filter doesn’t help if you still chain a second indexing step onto its result; the row and column selectors have to go into the same .loc[...] call.

5. A function receives a DataFrame, filters it, mutates the filtered subset, and returns nothing — the caller expects the original DataFrame to be updated in place. Rewrite it so it actually works, using what this lesson covered.

   Solution

   import pandas as pd
   
   def zero_out_negatives(df):
       df.loc[df["x"] < 0, "y"] = 0  # mutates df in place, no chain
   
   data = pd.DataFrame({"x": [1, -2, 3, -4], "y": [10, 20, 30, 40]})
   zero_out_negatives(data)
   print(data)

   $ python practice_chain5.py
      x   y
   0  1  10
   1 -2   0
   2  3  30
   3 -4   0

   The original version likely did subset = df[df["x"] < 0]; subset["y"] = 0 internally — the same silent two-step failure, just hidden inside a function call where it’s even harder to spot.