Menu

Earn Premium with Referrals

Invite your friends and earn Premium rewards through our referral program.

See how it works and start inviting friends.

Part 5: Concurrency & Top Gotchas
PYTHON

Part 5: Concurrency & Top Gotchas

Understand the Python GIL, threads versus processes, asynchronous programming, and common Python concurrency gotchas.

1. GIL — the headline

  • GIL (Global Interpreter Lock): a single lock allowing only one thread to execute Python bytecode at a time.
  • Consequence: pure-Python threads do not run in parallel on multiple cores.
  • Where it doesn’t matter: I/O-bound work (the interpreter releases GIL during blocking syscalls) and heavy C/Python extensions that release the GIL (numpy, etc.).
  • Where it hurts: CPU-bound pure-Python threads → no speedup across cores.

2. Threads vs Processes

threadingmultiprocessing
Concurrencyyesyes
ParallelismGIL-limitedyes (separate interpreters)
Memorysharedseparate (pickled IPC)
Best forI/O-boundCPU-bound
Costone processprocess spawn overhead
  • multiprocessing.Pool maps work across cores: Pool(4).map(func, data).
  • concurrent.futures: unified API — ThreadPoolExecutor vs ProcessPoolExecutor with Futures.

Gotcha on shared state: multiprocessing copies memory — a change in a child isn’t visible in the parent unless you use Queue/Pipe/Value/Array.

3. async / await

  • asyncio — single-threaded, cooperative multitasking for I/O-bound code.
  • async def coroutine, await suspends on blocking I/O, asyncio.run(...) drives the loop.
  • Tasks run concurrently, not in parallel; no preemption — blocking code (e.g., time.sleep, CPU work) blocks the whole loop.
import asyncio

async def ping(name):
    await asyncio.sleep(0.1)
    return name

async def main():
    results = await asyncio.gather(ping("a"), ping("b"))
    # concurrent; not parallel
  • asyncio vs threads: async/await — no OS threads/locks; but requires non-blocking code throughout. Threads are preemptive; async is cooperative.

4. Thread safety rules

  • Python + +=/x = x+1 is not atomic — read-modify-write can be interleaved.
  • Use threading.Lock around shared mutations.
  • queue.Queue is thread-safe for producer-consumer.
  • threading.local() for per-thread storage.

5. The gotcha sheet (interview gold)

  1. Mutable default args — shared across calls (def f(x=[])).
  2. Late binding closures / lambdas — loop var captured by reference.
  3. is vs == / small-int caching256 is 257 False, 257 False by identity.
  4. Integer division-7 // 2 = -4; -7 % 3 = 2.
  5. String + int — concatenation raises TypeError; conversion required.
  6. Chained assignmenta = b = [] shares the same list.
  7. Boolean trapbool([]) is False; bool(0) is False.
  8. Copy vs deepcopy — shallow share.
  9. tuple unhashable inner listhash((1, [])) → TypeError.
  10. return in a generatorStopIteration vs return triggering.
  11. String repetition vs addition"ab" * 3"ababab"; "ab" + 3TypeError.
  12. sorted returns a new list; .sort() sorts in place — know which one you’re calling.
  13. range vs list — lazy vs eager.
  14. raise vs assert — production raising vs debug-only assertion.
  15. __init__ vs __new__ — initializer vs constructor.

6. Performance quick hits

  • Favor built-ins and list-comprehensions over hand loops (C-accelerated).
  • if x in set vs in list — O(1) vs O(n).
  • Generator expressions for large one-shot pipelines.
  • Profile first — don’t micro-opt (the story is in cProfile/timeit).

7. Final interview patterns

Python interviews reward fundamentals: name the GIL, nail is vs ==, mutable defaults, and if __name__ == "__main__". Be ready for a time complexity question (list vs set/dict membership) and one asyncio overview.

My Private Notes

Notes are auto-saved locally to this device.