You wrote a small script on your Mac. It reads a notes.txt, does something tidy with it, prints the result. It works every time. You send it to a friend on Windows, and ten minutes later they send back a screenshot:
UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 20: character maps to <undefined>
The traceback — Python's printout when a program crashes, the block of text pointing at the line that broke — points at your open() line. Nothing about that line looks wrong, because it isn't wrong on your machine. I know that feeling, and I'm going to take the error apart so you can fix it for good, not patch it.
Here is the fix in one line, so you have it before the explanation: name the encoding every time you open a file.
with open("notes.txt", encoding="utf-8") as f: notes = f.read()
That is the whole cure. Writing it costs one short argument per open(); not writing it costs your friend an evening and you a screenshot. The rest of this piece is why it works, and the one case where you reach for a different table.
What the error is actually saying
A file on disk is just bytes: numbers from 0 to 255. Nothing in the file says "this byte is an e with an accent." Something has to decide which number stands for which letter. That decision is the encoding.
UTF-8 is one such decision, and it's the one most tools on your Mac write. In UTF-8, the letter é is two bytes. The em dash is three.
cp1252 is a different decision, older, used by Windows for decades. In cp1252, é is one byte, and some byte values mean nothing at all.
'charmap' in the error is Python telling you it is reading with a one-byte-per-letter table, and byte 0x9d has no letter in that table. So the error is not "your file is broken." It's "two machines disagree about the table."
Reproduce it in a minute
You don't need a Windows PC to see the mechanism. Write a file the way your Mac does, then read it with the Windows table spelled out:
# make_notes.py text = "café — “hello”\n" # accent, em dash, curly quotes with open("notes.txt", "w", encoding="utf-8") as f: f.write(text) print(open("notes.txt", "rb").read())
Run it and you get the raw bytes:
b'caf\xc3\xa9 \xe2\x80\x94 \xe2\x80\x9chello\xe2\x80\x9d\n'
Look at the é: two bytes, \xc3\xa9. Now read the same bytes with the Windows table:
print(open("notes.txt", encoding="cp1252").read())
On the Python 3.15 beta I ran, that raised exactly the error from the top of this piece: 'charmap' codec can't decode byte 0x9d in position 20. That 0x9d is the last byte of the closing curly quote. The Windows table has no letter for it.
Not every file crashes. A file with only café in it doesn't fail under the Windows table, it quietly returns this:
café
That is worse in one way, because nothing tells you it's wrong. I decoded the UTF-8 bytes for café with cp1252 on Python 3.11 and got that é debris back. If your friend's output has strange à and †characters in it, this is why.
Why your Mac never complained
When you call open() with no encoding, Python picks one for you. Historically that pick came from the machine's locale, its language-and-region settings. On a Mac or a Linux box the locale is virtually always UTF-8, so the pick is UTF-8 and your file reads fine. On a Windows machine in many regions the locale pick is cp1252.
Same script, same file, same bytes. Different table at the other end.
This is the honest scope, and I want to be careful about it. This bites Windows beginners. If you only run your code on a Mac or Linux, you will probably never see this error from a bare open(). I checked on the Linux box I write on: bare open() reads that file perfectly, and on this machine it never chose cp1252 in the first place.
What Python 3.15 changes
Python 3.15 changes the default. PEP 686 — a Python Enhancement Proposal (that's what "PEP" means) titled "Make UTF-8 Mode Default," with status Final against version 3.15 — turns UTF-8 mode on by default, so a bare open() no longer leans on the machine's locale: with no encoding given, it reads and writes UTF-8 everywhere.
I want to be exact about where that comes from, because it's the kind of claim this site exists to get right. That behaviour is what PEP 686 and the Python 3.15 release notes document — not something I ran on a stable 3.15 myself. The interpreters I actually ran here are 3.15.0b4 (a pre-release) and 3.11, both on Linux, so I'm citing the spec and the notes for the flip and reproducing the underlying mechanism on the versions I had. Not sure which Python you're on? Run python --version (or python3 --version) and read the number.
One honest note on what that means for testing. I write on Linux, where the locale was already UTF-8, so a bare open() here read the file fine before and after — I can't stage the Windows flip on my own screen. But per PEP 686 and the 3.15 notes it's the Windows default that moves: from 3.15 on, a Windows machine's bare open() reads your Mac-written notes.txt as UTF-8 instead of reaching for cp1252. The sent-from-a-Mac case gets fixed.
The flip cuts both ways
Now the part the fix-this-error articles skip. A new default doesn't know which files you own.
Say your friend has an old file, saved by a Windows tool in its "ANSI" format, which means cp1252. A bare open() that used to read it fine will now try UTF-8, and an accent in it can break. I reproduced that failure on my own bytes. I wrote a file as cp1252, then read it as UTF-8:
open("old.txt", "w", encoding="cp1252").write("café — “hello”\n") open("old.txt", encoding="utf-8").read()
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 3: invalid continuation byte
Byte 0xe9 is the single-byte é of cp1252, and UTF-8 refuses it. That's what a Windows user would see after the switch, reading an old file.
So the change is both cure and cause. It cures files that came from a Mac or Linux. It can newly break files written by older Windows tools.
You have two fixes for the second case. Say what the file really is:
open("old.txt", encoding="cp1252").read()
Or open it once in an editor, re-save it as UTF-8, and stop carrying two tables around.
Why this matters
Neither fix needs a version number. Both say the same thing: you tell Python which table the file uses, instead of letting the machine guess.
That is the habit from the top of this piece, and it's the only one correct on every version, every operating system, and every friend's laptop. The same applies when you write: if you save with a bare open("out.txt", "w"), the guess happens on the way out too, and the next person to read it inherits your machine's opinion. Spell out the encoding there as well.
Finding the open() calls you already wrote
You have old scripts. Don't read them all. Ask Python to point at the guesses. Put this one line in a file called your_script.py:
open("notes.txt").read()
Then run it in your terminal with the warning switched on:
python -X warn_default_encoding your_script.py
Python 3.15.0b4 printed two lines:
/home/dmitri/your_script.py:1: EncodingWarning: 'encoding' argument not specified open("notes.txt").read()
The first line names the file and the line number, and the second echoes the offending line of your code. The path will be your own; I ran it from a different folder than you will. If you run a one-liner with python -c instead of a file, the location reads <string>:1: because there is no file to name, so don't be thrown if you see that form elsewhere.
Every warning is a line that depends on a guess. That flag exists on Python 3.10 and later, and I ran a script file against it on 3.11 and on the 3.15 beta, same two-line warning on both. One thing to know: it only fires when you leave encoding out. On my Linux box it fired even though the read worked, so it tells you where you are guessing, not where you are failing.
A quick way to tell which table a file used
If you're handed a file and don't know where it came from, don't guess, test. Read it as UTF-8 first. UTF-8 is strict, so a file that wasn't written as UTF-8 usually fails loudly, as the old.txt example did. If it fails, try cp1252 and look at the words. If you see real words with real accents, you've found the table. If you see à and †debris, you read UTF-8 bytes with the wrong table and should go back.
This is a diagnosis, not a guarantee. A short file with no accents reads the same under both, because plain English letters are the same bytes in each table. Those are the files that make you think there's no problem, right up until someone's name has an accent in it.
Your next step
Open the oldest script you've shared with anyone. Run it once with python -X warn_default_encoding. For every EncodingWarning, add encoding="utf-8", unless the file came from an older Windows tool, in which case check it first by reading it both ways and seeing which one gives you real words.
Then send it to the friend with the Windows laptop. If you get a screenshot back, it won't be this one.
And if you are that friend — handed a file, staring at the charmap error, not the one who wrote the script — the move is the same habit pointed the other way: the file you were sent is almost certainly UTF-8, so open it with encoding="utf-8" on your end and it reads clean.