Stream: helpdesk (published)

Topic: Reproducibiliy of `rand()`

view this post on Zulip Nils (Dec 06 2023 at 09:11):

I'm aware that rand() is not guaranteed to give the same output even when using the same seed in general, and that people who need this have to use StableRNGs.jl. I'm wondering though whether there are cases where I should expect the same outputs? E.g. if I use the same Julia binary on the same OS would that be expected to yield the same numbers?

If someone wants to try, I'm on 1.10-rc2 on Windows 10:

julia> using Random

julia> Random.seed!(123)
TaskLocalRNG()

julia> rand()
0.521213795535383

view this post on Zulip Sukera (Dec 06 2023 at 09:38):

it definitely should give the same output for the same seed (on the same version)

view this post on Zulip Sukera (Dec 06 2023 at 09:39):

I think this even goes as far as being the same on the same architecture

view this post on Zulip Sukera (Dec 06 2023 at 09:39):

julia> using Random

julia> Random.seed!(123)
TaskLocalRNG()

julia> rand()
0.521213795535383

julia> versioninfo()
Julia Version 1.11.0-DEV.822
Commit 528fefb66a7* (2023-11-12 12:24 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: 24 × AMD Ryzen 9 7900X 12-Core Processor
  WORD_SIZE: 64
  LLVM: libLLVM-15.0.7 (ORCJIT, znver3)
  Threads: 34 on 24 virtual cores
Environment:
  JULIA_PKG_USE_CLI_GIT = true

view this post on Zulip Sukera (Dec 06 2023 at 09:40):

the issue with relying on this to never change is that it may change between versions & architectures; generally though, for the same julia version it should give the same output.

view this post on Zulip Jakob Nybo Nissen (Dec 06 2023 at 09:59):

Does it really depend on the arch though? How? Does it use FMA?

view this post on Zulip Sukera (Dec 06 2023 at 10:52):

I'm pretty sure it shouldn't, but I don't think Julia actually guarantees it

view this post on Zulip Jakob Nybo Nissen (Dec 06 2023 at 10:58):

Hard to tell: The reproducibility section only mentions Julia version changes

view this post on Zulip Sukera (Dec 06 2023 at 11:00):

that section doesn't give a whole lot guarantees at all anyway

view this post on Zulip Sukera (Dec 06 2023 at 11:00):

from the way the RNGs are currently implemented, it should be consistent across architectures

view this post on Zulip Sukera (Dec 06 2023 at 11:01):

Julia just doesn't guarantee that to be the case

view this post on Zulip Sukera (Dec 06 2023 at 11:01):

e.g. I'm not sure you're guaranteed to get the same result on a GPU, which has a different default RNG

view this post on Zulip Nils (Dec 06 2023 at 11:41):

Cool thanks - it sounds to me like I'm okay to share a notebook that includes versioninfo() and Random.seed!(123) in the expectation that someone would be able to get the same outputs.

view this post on Zulip Lilith Hafner (Dec 06 2023 at 15:46):

Currently, in 1.9 and 1.10, specifying the exact Julia version is sufficient. However, IMO by doing this you are depending on internals (i.e. there is no guarantee that this will still work when you and the folks reproducing your code are on 1.12)

view this post on Zulip Nils (Dec 06 2023 at 16:27):

Sure, but that's expected. I'm just after having a historical record that says "this sample was drawn with Julia 1.10rc-2 on Windows 10, using the random seed 123" so that someone else can download the same Julia version and get the same results with the same seed.

view this post on Zulip Mason Protter (Dec 06 2023 at 17:07):

Seems like that's what StableRNGs.jl is for

view this post on Zulip Nils (Dec 06 2023 at 18:00):

Yes as I said originally that's the correct solution, I just wondered under which circumstances I can go without it.

view this post on Zulip Mosè Giordano (Dec 06 2023 at 21:19):

Or https://github.com/wildart/Xkcd221 for even stronger reproducibility guarantees

Last updated: Nov 06 2024 at 04:40 UTC