NumPy vs PyTorch dtype Pitfalls: Memory Sharing, Integer dtypes, and float16 Precision Loss

Introduction

NumPy’s default floating-point type is float64 (double precision), while PyTorch’s default is float32 (single precision). This seemingly minor difference has real consequences:

• Memory usage doubles (8 bytes per element vs. 4 bytes)
• GPU throughput (most GPUs are optimized for float32 and below; float64 is dramatically slower)
• Numerical stability of gradient computation and optimization

On top of that, torch.tensor() and torch.from_numpy() — the two most common ways to turn a NumPy array into a PyTorch tensor — look similar but behave completely differently, and integer dtypes and default-dtype changes bring their own pitfalls. This article walks through each of these with actually executed code and output.

1. The Default dtype Mismatch Between NumPy and PyTorch

import numpy as np
import torch

np.random.seed(0)
x = np.random.rand(5)
print(x)
print(x.dtype)

torch_x = torch.tensor(x)
print(torch_x)

[0.5488135  0.71518937 0.60276338 0.54488318 0.4236548 ]
float64
tensor([0.5488, 0.7152, 0.6028, 0.5449, 0.4237], dtype=torch.float64)

The array produced by np.random.rand() is float64. Because torch.tensor() preserves the source array’s dtype, the resulting tensor is also float64. Since PyTorch neural network weights default to float32, this can lead to dtype mismatch errors, or silently slower computation running in double precision.

torch_x32 = torch.tensor(x.astype(np.float32))
print(torch_x32)

print("float64 tensor element size:", torch.tensor(x).element_size(), "bytes")
print("float32 tensor element size:", torch_x32.element_size(), "bytes")

tensor([0.5488, 0.7152, 0.6028, 0.5449, 0.4237])
float64 tensor element size: 8 bytes
float32 tensor element size: 4 bytes

To explicitly unify the dtype, casting on the NumPy side with astype(np.float32) before creating the tensor is the safest approach. element_size() confirms that the per-element byte count is indeed halved.

Why the precision difference matters

An IEEE 754 float is made of a sign bit, exponent bits, and mantissa bits; the number of mantissa bits \(p\) determines the relative rounding error (machine epsilon), \(\epsilon \approx 2^{-p}\) . We can check this directly with torch.finfo:

import torch

for dt in [torch.float16, torch.bfloat16, torch.float32, torch.float64]:
    fi = torch.finfo(dt)
    print(f"{dt}: bits={fi.bits:>2}  eps={fi.eps:.3e}  max={fi.max:.3e}")

torch.float16: bits=16  eps=9.766e-04  max=6.550e+04
torch.bfloat16: bits=16  eps=7.812e-03  max=3.390e+38
torch.float32: bits=32  eps=1.192e-07  max=3.403e+38
torch.float64: bits=64  eps=2.220e-16  max=1.798e+308

float32 has a relative precision of about \(1.2 \times 10^{-7}\) (23 mantissa bits), while float64 reaches about \(2.2 \times 10^{-16}\) (52 mantissa bits). Most deep learning tasks are fine with float32 precision, and since float64 roughly doubles both memory footprint and compute cost, explicitly casting to float32 is the standard practice.

2. torch.tensor() vs. torch.from_numpy(): A Crucial Difference

Both functions turn a NumPy array into a tensor, but they differ fundamentally in whether they share memory with the source array:

• torch.from_numpy(arr) — returns a tensor that shares memory with the original array (no copy)
• torch.tensor(arr) — always copies the data into a new tensor

This is easy to miss and is a classic source of bugs where the original data gets modified unintentionally. Let’s confirm it directly.

import numpy as np
import torch

# --- torch.from_numpy(): shares memory ---
a = np.array([1.0, 2.0, 3.0], dtype=np.float32)
t = torch.from_numpy(a)
print("before:", a, t)

t[0] = 100.0  # modify the tensor in-place
print("after tensor edit -> numpy array:", a)

a[1] = -999.0  # modify the numpy array in-place
print("after numpy edit  -> tensor:     ", t)

print("share memory (from_numpy):", np.shares_memory(a, t.numpy()))

before: [1. 2. 3.] tensor([1., 2., 3.])
after tensor edit -> numpy array: [100.   2.   3.]
after numpy edit  -> tensor:      tensor([ 100., -999.,    3.])
share memory (from_numpy): True

Simply modifying an element of tensor t changes the original NumPy array a too — and the reverse direction (modifying the NumPy array changes the tensor) also holds. np.shares_memory() confirms they physically reference the same memory.

# --- torch.tensor(): copies the data ---
b = np.array([1.0, 2.0, 3.0], dtype=np.float32)
u = torch.tensor(b)
u[0] = 100.0
print("after tensor edit -> numpy array (torch.tensor case):", b)
print("tensor itself:", u)
print("share memory (torch.tensor):", np.shares_memory(b, u.numpy()))

after tensor edit -> numpy array (torch.tensor case): [1. 2. 3.]
tensor itself: tensor([100.,   2.,   3.])
share memory (torch.tensor): False

torch.tensor(), in contrast, makes a copy: modifying the tensor leaves the original NumPy array b untouched.

Practical guidance: use torch.from_numpy() when you want to avoid the conversion cost on large, read-only arrays; use torch.tensor() when you plan to reuse the original array afterward or want to avoid unintended side effects. Note also that passing a read-only (writeable=False) NumPy array to torch.from_numpy() triggers a warning, since writing through the resulting tensor would be undefined behavior:

a_ro = np.array([1.0, 2.0, 3.0], dtype=np.float32)
a_ro.setflags(write=False)
t_ro = torch.from_numpy(a_ro)
print(t_ro)

UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /Users/runner/work/pytorch/pytorch/torch/csrc/utils/tensor_numpy.cpp:219.)
tensor([1., 2., 3.])

3. Integer dtype Pitfalls: Platform Dependence and Indexing Ops

NumPy’s default integer dtype is int64 on Linux/macOS, but on Windows it can default to int32 (tied to the platform’s C long size). Meanwhile, some PyTorch APIs — such as nn.functional.one_hot and various embedding/indexing operations — strictly require int64 (torch.long) indices. This mismatch causes the classic “works on my machine, breaks on Windows” portability bug. Let’s reproduce it by passing an int32 index explicitly.

import numpy as np
import torch
import torch.nn.functional as F

print("numpy default int dtype on this platform:", np.array([1, 2, 3]).dtype)

# On Windows, np.array([...]) can default to int32
idx_np_int32 = np.array([0, 2, 1], dtype=np.int32)
idx_np_int64 = np.array([0, 2, 1], dtype=np.int64)

idx64 = torch.from_numpy(idx_np_int64)
idx32 = torch.from_numpy(idx_np_int32)
print("idx64 dtype:", idx64.dtype, "/ idx32 dtype:", idx32.dtype)

out64 = F.one_hot(idx64, num_classes=5)
print("F.one_hot(int64) OK:\n", out64)

try:
    out32 = F.one_hot(idx32, num_classes=5)
    print("F.one_hot(int32) OK:\n", out32)
except Exception as e:
    print(f"F.one_hot(int32) raised {type(e).__name__}: {e}")

numpy default int dtype on this platform: int64
idx64 dtype: torch.int64 / idx32 dtype: torch.int32
F.one_hot(int64) OK:
 tensor([[1, 0, 0, 0, 0],
        [0, 0, 1, 0, 0],
        [0, 1, 0, 0, 0]])
F.one_hot(int32) raised RuntimeError: one_hot is only applicable to index tensor of type LongTensor.

Passing an int32 index raises a RuntimeError. When building index arrays on the NumPy side, explicitly specify dtype=np.int64 to avoid platform dependence, or call .long() (equivalent to .to(torch.int64)) on the PyTorch side before using the tensor as an index.

4. torch.set_default_dtype() Only Affects Tensors Created Afterward

torch.set_default_dtype() changes the default floating-point dtype, but this change does not retroactively apply to tensors or NumPy arrays that already exist. This “going forward, not retroactive” distinction is a common source of confusion.

import numpy as np
import torch

print("default dtype before:", torch.get_default_dtype())

existing = torch.tensor([1.0, 2.0, 3.0])
print("existing tensor dtype (created before change):", existing.dtype)

torch.set_default_dtype(torch.float64)
print("default dtype after set_default_dtype(float64):", torch.get_default_dtype())

new_tensor = torch.tensor([1.0, 2.0, 3.0])
print("new tensor dtype (created after change):", new_tensor.dtype)

print("existing tensor dtype (unchanged):", existing.dtype)

np_arr_f32 = np.array([1.0, 2.0, 3.0], dtype=np.float32)
print("numpy array dtype (unaffected by set_default_dtype):", np_arr_f32.dtype)

from_np = torch.from_numpy(np_arr_f32)
print("torch.from_numpy(np_arr_f32).dtype (still float32, ignores default):", from_np.dtype)

torch.set_default_dtype(torch.float32)  # reset

default dtype before: torch.float32
existing tensor dtype (created before change): torch.float32
default dtype after set_default_dtype(float64): torch.float64
new tensor dtype (created after change): torch.float64
existing tensor dtype (unchanged): torch.float32
numpy array dtype (unaffected by set_default_dtype): float32
torch.from_numpy(np_arr_f32).dtype (still float32, ignores default): torch.float32

Three takeaways:

1. existing, created before the set_default_dtype() call, is unaffected
2. Only tensors created after the call — with torch.tensor(python_list) and no explicit dtype — pick up the new default
3. NumPy array dtypes, and the dtype torch.from_numpy() inherits from its source array, are completely independent of PyTorch’s default dtype setting

Bugs where “I thought I switched the whole model to float64, but some layers stayed float32” almost always trace back to misunderstanding points 2 and 3.

5. float16/bfloat16 and Precision Loss in Reductions (sum/mean)

float16 (half precision) and bfloat16 are increasingly used to speed up GPU training, but reduction operations like sum and mean can lose substantial precision as rounding error accumulates. Let’s confirm this on a one-million-element random array.

import torch

torch.manual_seed(0)

n = 1_000_000
x32 = torch.rand(n, dtype=torch.float32)
x16 = x32.to(torch.float16)
xb16 = x32.to(torch.bfloat16)

sum32 = x32.sum().item()
sum16 = x16.sum().item()
sumb16 = xb16.sum().item()

print("sum (float32): ", sum32)
print("sum (float16): ", sum16)
print("sum (bfloat16):", sumb16, " rel_err:", abs(sum32 - sumb16) / sum32)

sum (float32):  500268.03125
sum (float16):  inf
sum (bfloat16): 499712.0  rel_err: 0.0011114666843904989

The float16 sum is inf — infinity. This isn’t just accumulated rounding error; it’s an outright overflow. float16’s maximum representable value is about 65504 (torch.finfo(torch.float16).max), and the sum of a million elements averaging around 0.5 reaches roughly 500,000 — well past that limit, so it overflows to inf partway through. bfloat16, meanwhile, has only 7 mantissa bits (coarser precision than float16, with about 8x larger eps), but its exponent uses the same 8 bits as float32, giving it the same wide range (max ≈ \(3.39 \times 10^{38}\) , essentially identical to float32). As a result it doesn’t overflow, landing at only about 0.11% relative error.

This tradeoff — float16 has a narrow range and overflows easily, bfloat16 has a wide range but coarser precision — is exactly why mixed-precision training (AMP) with float16 needs loss/gradient scaling (e.g. GradScaler).

Sweeping the array size makes the pre/post-overflow behavior clearer.

import numpy as np
import torch

torch.manual_seed(0)

sizes = [10, 30, 100, 300, 1_000, 3_000, 10_000, 30_000, 60_000, 90_000,
         100_000, 110_000, 120_000, 125_000, 128_000, 130_000, 131_000,
         135_000, 150_000, 300_000, 1_000_000]

x32_full = torch.rand(max(sizes), dtype=torch.float32)

for n in sizes:
    x32 = x32_full[:n]
    x16 = x32.to(torch.float16)
    s32 = x32.sum().item()
    s16 = x16.sum().item()
    abs_err = abs(s32 - s16)
    rel_err = abs_err / abs(s32) if np.isfinite(s16) else float("inf")
    print(f"n={n:>8}  sum32={s32:>12.4f}  sum16={s16!s:>12}  rel_err={rel_err}")

n=      10  sum32=      4.9010  sum16=  4.90234375  rel_err=0.0002822517364416764
n=      30  sum32=     13.9841  sum16=   13.984375  rel_err=2.1959499317959776e-05
n=     100  sum32=     48.8419  sum16=    48.84375  rel_err=3.686456272067971e-05
n=     300  sum32=    142.8603  sum16=     142.875  rel_err=0.00010296403991586856
n=    1000  sum32=    500.9248  sum16=       501.0  rel_err=0.00015011297463480633
n=    3000  sum32=   1491.0660  sum16=      1491.0  rel_err=4.429048565868344e-05
n=   10000  sum32=   5002.7432  sum16=      5004.0  rel_err=0.0002512293548324757
n=   30000  sum32=  15080.9707  sum16=     15080.0  rel_err=6.436609049302483e-05
n=   60000  sum32=  30031.0371  sum16=     30032.0  rel_err=3.20631825498763e-05
n=   90000  sum32=  45035.7656  sum16=     45056.0  rel_err=0.00044929568131439975
n=  100000  sum32=  50027.6211  sum16=     50016.0  rel_err=0.0002322935509610277
n=  110000  sum32=  55062.0156  sum16=     55072.0  rel_err=0.00018132963144681466
n=  120000  sum32=  60065.6875  sum16=     60064.0  rel_err=2.8094242657257524e-05
n=  125000  sum32=  62564.1562  sum16=     62560.0  rel_err=6.643180774934529e-05
n=  128000  sum32=  64081.5625  sum16=     64064.0  rel_err=0.00027406479047698
n=  130000  sum32=  65099.9414  sum16=     65088.0  rel_err=0.00018343190473061702
n=  131000  sum32=  65610.5000  sum16=         inf  rel_err=inf
n=  135000  sum32=  67610.8750  sum16=         inf  rel_err=inf
n=  150000  sum32=  75086.6797  sum16=         inf  rel_err=inf
n=  300000  sum32= 150034.8750  sum16=         inf  rel_err=inf
n= 1000000  sum32= 500268.0312  sum16=         inf  rel_err=inf

Below n=131,000, the relative error fluctuates irregularly around \(10^{-5}\) –\(10^{-4}\) . But right around n=131,000, the float32 sum crosses float16’s maximum representable value (about 65504), and the float16 sum abruptly diverges to inf. This isn’t a gradual precision decay — it’s a threshold effect where the computation completely breaks down past a certain size, and it’s a real risk when summing or averaging large float16 tensors in practice. This is exactly why most AMP implementations automatically upcast reduction operations like sum to float32 internally, or compute the loss in float32.

6. A Recent Development: NumPy 2.0’s Scalar Promotion Rules (NEP 50)

A recent, relevant change in this space is NEP 50 (Promotion rules for Python scalars), introduced in NumPy 2.0. It changes how Python’s built-in int/float/complex types combine with NumPy arrays in type promotion. A representative example: np.float32(3) + 3.0 used to be upcast to float64 under NumPy 1.x, but under NumPy 2.0 and later it stays float32. This moves NumPy’s dtype-propagation behavior closer to PyTorch’s philosophy (preserve the existing tensor’s dtype unless told otherwise), reducing accidental dtype changes when mixing NumPy and PyTorch. That said, code that implicitly relied on the old float64 upcasting behavior may see behavioral differences after migrating to NumPy 2.0.

Summary

• NumPy defaults to float64; PyTorch defaults to float32. Without explicitly specifying dtype during conversion, you risk unintended precision, memory, and speed consequences
• torch.from_numpy() shares memory with the source array; torch.tensor() copies it. The former is fast but can cause unintended side effects on the original data
• NumPy’s default integer dtype is platform-dependent (can be int32 on Windows), and some PyTorch APIs (e.g. F.one_hot) require int64, causing a RuntimeError if the wrong dtype is passed
• torch.set_default_dtype() only affects tensors created afterward — it has no effect on existing tensors or NumPy arrays
• float16 has a narrow representable range and overflows easily; bfloat16 has a wide range but coarser precision. Consider upcasting to float32 for large sum/mean reductions
• NumPy 2.0’s NEP 50 scalar promotion change reduces implicit dtype drift, but be aware of behavioral differences when migrating existing code

With these in mind, the best defense when combining NumPy and PyTorch is to make explicit dtype specification a habit at every conversion boundary — it’s the shortest path to a reproducible, bug-resistant pipeline.