OS-based rng access

rand.rkt

#lang racket/base
(require racket/contract/base  "unix_rand.rkt" "windows_rand.rkt" (only-in file/sha1 bytes->hex-string))
(provide (contract-out [crypto-random-bytes (-> exact-positive-integer? (or/c bytes? string?))]
                       [crypto-random (-> exact-positive-integer? exact-positive-integer?)]))
                       
(define bytes->number (compose (lambda (s) (string->number s 16)) bytes->hex-string))

; (: crypto-random-bytes (-> Positive-Integer Bytes))
; returns n random bytes from the os.
(define (crypto-random-bytes n)
  (case (system-type 'os)
    [(unix macosx) (crypto-random-unix-bytes n)]
    [(windows) (crypto-random-windows-bytes n)]
    [else (raise (make-exn:fail:unsupported
                  "Only UNIX, OS X, and Windows XP or greater are currently supported"
                  (current-continuation-marks)))]))

; (: crypto-random (-> Positive-Integer Nonnegative-Integer))
; returns a random n byte nonnegative-integer
(define (crypto-random n)
  (bytes->number (crypto-random-bytes n)))

unix_rand.rkt

#lang racket/base
(provide crypto-random-unix-bytes)

(define (check-urandom-exists)
  (unless (file-exists? "/dev/urandom")
    (raise (make-exn:fail:filesystem
            "/dev/urandom does not exist"
            (current-continuation-marks)))))

; (: crypto-random-unix-bytes (-> Positive-Integer Bytes))
(define (crypto-random-unix-bytes n)
  (check-urandom-exists)
  (define urandom (open-input-file "/dev/urandom"))
  (read-bytes n urandom))

windows_rand.rkt

#lang racket/base
(provide crypto-random-windows-bytes)

(require ffi/com 
         ffi/unsafe 
         ffi/unsafe/define 
         ffi/winapi)

(define-ffi-definer define-advapi (and (eq? (system-type) 'windows) (ffi-lib "Advapi32.dll"))
  #:default-make-fail make-not-available)

; supposed to be the same csprng as CryptGenRand, but with less overhead
; see Microsoft security dev Michael Howard: http://blogs.msdn.com/b/michael_howard/archive/2005/01/14/353379.aspx
; this is for Windows XP and later only, but I doubt that's a problem
(define-advapi SystemFunction036 
  (_fun #:abi winapi _pointer _ulong -> _bool))

(define (check-SystemFunction036-exists)
  (unless SystemFunction036
    (raise (make-exn:fail:unsupported
            "Unable to load RtlGenRandom (SystemFunction036) from Advapi32.dll"
            (current-continuation-marks)))))

; (: crypto-random-windows-bytes (-> Positive-Integer Bytes))
(define (crypto-random-windows-bytes n)
  (check-SystemFunction036-exists)
  (define rand-bytes-buf (make-bytes n))
  (if (SystemFunction036  rand-bytes-buf n)
      rand-bytes-buf
      (raise (make-exn:fail
              "SystemFunction036 failed to generate bytes"
              (current-continuation-marks)))))

reworked error handling in line with @jackfirth's suggestions. Hopefully I did that correctly.
Added crypto-random which returns an integer as opposed to a byte string
added optional argument to crypto-random-bytes that allows hex-string output - is this a good idea?
[and is there a better way to do bytes->number conversion?]

More minor thoughts:

• You could make a (raise-unsupported-error message) function in a private helper module that does (raise (make-exn:fail:unsupported message (current-continuation-marks))) to eliminate some duplication and increase readability.
• Instead of passing a flag to allow hex string output, it would be easier to have a crypto-random-hex-string function. Though that's probably not too useful to provide since it's just (compose bytes->hex-string crypto-random-bytes).
• For implementing crypto-random, it'd be more useful to do it in terms of a bytes->number function. I don't know a better way to make that function than by doing bytes->hex-string followed by string->number. Hand rolling your own in terms of bytes->list is a possibility but it would be messy and doesn't really belong with this feature.
• crypto-random's input n should behave the same way as random's and be an upper bound on the result number, not the number of bytes to use to generate the result number. This is trickier, but it's worth it since it makes crypto-random a drop-in replacement for random (in the one-argument case).

The exception raising looks good to me. I find it strange there isn't a built in bytes->number function to use. Maybe a separate PR should add that? Unless there's some good reason it's not provided.

I decided to remove the hex string option. bytes->hex-string makes it trivial for someone to convert the output of crypto-random-bytes into a hex string if they need it, so it doesn't warrant the added complexity.

a bytes->number would definitely be a useful addition to the bytes pkg.

your point about crypto-random's interface is spot-on. I'll have to think of how best to do this.