Stream: helpdesk (published)

Topic: Reproducibiliy of `rand()`

Nils (Dec 06 2023 at 09:11):

I'm aware that rand() is not guaranteed to give the same output even when using the same seed in general, and that people who need this have to use StableRNGs.jl. I'm wondering though whether there are cases where I should expect the same outputs? E.g. if I use the same Julia binary on the same OS would that be expected to yield the same numbers?

If someone wants to try, I'm on 1.10-rc2 on Windows 10:

julia> using Random

julia> Random.seed!(123)
TaskLocalRNG()

julia> rand()
0.521213795535383

Sukera (Dec 06 2023 at 09:38):

it definitely should give the same output for the same seed (on the same version)

Sukera (Dec 06 2023 at 09:39):

I think this even goes as far as being the same on the same architecture

Sukera (Dec 06 2023 at 09:39):

julia> using Random

julia> Random.seed!(123)
TaskLocalRNG()

julia> rand()
0.521213795535383

julia> versioninfo()
Julia Version 1.11.0-DEV.822
Commit 528fefb66a7* (2023-11-12 12:24 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: 24 × AMD Ryzen 9 7900X 12-Core Processor
  WORD_SIZE: 64
  LLVM: libLLVM-15.0.7 (ORCJIT, znver3)
  Threads: 34 on 24 virtual cores
Environment:
  JULIA_PKG_USE_CLI_GIT = true

Sukera (Dec 06 2023 at 09:40):

the issue with relying on this to never change is that it may change between versions & architectures; generally though, for the same julia version it should give the same output.

Jakob Nybo Nissen (Dec 06 2023 at 09:59):

Does it really depend on the arch though? How? Does it use FMA?

Sukera (Dec 06 2023 at 10:52):

I'm pretty sure it shouldn't, but I don't think Julia actually guarantees it

Jakob Nybo Nissen (Dec 06 2023 at 10:58):

Hard to tell: The reproducibility section only mentions Julia version changes

Sukera (Dec 06 2023 at 11:00):

that section doesn't give a whole lot guarantees at all anyway

Sukera (Dec 06 2023 at 11:00):

from the way the RNGs are currently implemented, it should be consistent across architectures

Sukera (Dec 06 2023 at 11:01):

Julia just doesn't guarantee that to be the case

Sukera (Dec 06 2023 at 11:01):

e.g. I'm not sure you're guaranteed to get the same result on a GPU, which has a different default RNG

Nils (Dec 06 2023 at 11:41):

Cool thanks - it sounds to me like I'm okay to share a notebook that includes versioninfo() and Random.seed!(123) in the expectation that someone would be able to get the same outputs.

Lilith Hafner (Dec 06 2023 at 15:46):

Currently, in 1.9 and 1.10, specifying the exact Julia version is sufficient. However, IMO by doing this you are depending on internals (i.e. there is no guarantee that this will still work when you and the folks reproducing your code are on 1.12)

Nils (Dec 06 2023 at 16:27):

Sure, but that's expected. I'm just after having a historical record that says "this sample was drawn with Julia 1.10rc-2 on Windows 10, using the random seed 123" so that someone else can download the same Julia version and get the same results with the same seed.

Mason Protter (Dec 06 2023 at 17:07):

Seems like that's what StableRNGs.jl is for

Nils (Dec 06 2023 at 18:00):

Yes as I said originally that's the correct solution, I just wondered under which circumstances I can go without it.

Mosè Giordano (Dec 06 2023 at 21:19):

Or https://github.com/wildart/Xkcd221 for even stronger reproducibility guarantees

Last updated: Nov 22 2024 at 04:41 UTC