Tag Archives: Python

Python Global Interpreter Lock (GIL)

Some notes on Python’s GIL:

  1. It is basically a mechanism in the CPython interpreter (the most common Python implementation) that ensures only one thread can execute Python bytecode at a time.
  2. The GIL prevents multiple threads from modifying reference counts at the same time, which could lead to memory corruption.
  3. This limits the performance gains you can achieve with multithreading for CPU-intensive tasks.
  4. But threads often spend time waiting for I/O operations (like network requests or file access), which can allow other threads to run even with the GIL. So, multithreading can still be beneficial for I/O-bound tasks.
  5. If you need true parallelism for CPU-bound tasks, use the multiprocessing module to create multiple processes, each with its own interpreter and GIL.
  6. Some libraries, like NumPy and Cython, release the GIL during certain operations, allowing better performance for multithreaded code.
  7. Alternative Python implementations like Jython and IronPython don’t have a GIL, but they have their own limitations and trade-offs.

Numpy Broadcasting

Broadcasting is a powerful feature in NumPy that allows you to perform arithmetic operations on arrays of different shapes. It essentially stretches the smaller array to match the dimensions of the larger array, enabling element-wise operations. This avoids the need for explicit looping and makes the code concise and efficient.

Let’s study it using some examples:

Example 1: Adding a scalar to a 2D array

Python

import numpy as np

scalar = 5
array = np.array([[1, 2, 3], [4, 5, 6]])

# Broadcasting adds the scalar to each element of the array
result = scalar + array
print(result)
Output:
[[6 7 8]
 [9 10 11]

In this example, the scalar 5 is added to each element of the 2D array array. Broadcasting automatically stretches the scalar to match the shape of the array, resulting in the output array.

Example 2: Multiplying a 1D array with a 2D array

vector = np.array([1, 2, 3])
array = np.array([[1, 2, 3], [4, 5, 6]])

# Broadcasting multiplies the vector element-wise with each row of the array
result = vector * array
print(result)
Output:
[[1 2 3]
 [4 10 18]]

In this example, the 1D array vector is element-wise multiplied with each row of the 2D array array. Broadcasting automatically stretches the 1D array to match the number of columns in the 2D array, resulting in the output array.

Example 3: Adding two arrays of different shapes

array1 = np.array([[1, 2, 3]])  # 1D array
array2 = np.array([[4], [5], [6]])  # 2D array

# Broadcasting stretches the 1D array to match the 2D array shape
result = array1 + array2
print(result)
Output:

[[5 6 7]
 [5 6 7]
 [5 6 7]]

In this example, the 1D array array1 is added to the 2D array array2. Broadcasting stretches the 1D array to match the shape of the 2D array, resulting in the output array.

Dot vs Element-wise multiplication

Dot Multiplication

Properties

  • It is performed via numpy.dot or using the @ operator.
  • It represents the traditional matrix multiplication.
  • Involves summing the products of corresponding elements in rows and columns.
  • Output shape depends on the input shapes:
  • For matrices A (m x n) and B (n x p), the output is (m x p).
  • For vectors, it produces a scalar (single value).

Example

import numpy as np
a = np.array([[1, 2], [3, 4]]) #  [[1 2]
                              #   [3 4]]

b = np.array([[5, 6], [7, 8]]) #  [[5 6]
                              #   [7 8]]

result = np.dot(a, b)  # Or result = a @ b
print(result)  # Output: [[19 22]
             #           [43 50]]

Element-wise multiplication

Properties

  • It is performed using numpy.multiply or using the * operator.
  • It implies multiplying corresponding elements of arrays directly.
  • Output shape matches the input shapes (if compatible).
  • Broadcasting rules apply for different-shaped arrays.

Example

a = np.array([1, 2, 3])
b = np.array([4, 5, 6]) 
result = np.multiply(a, b) # Or result = a * b
print(result) # Output: [ 4 10 18]