Monte Carlo methods (Fishman, 1996; Gentle, 1998; Robert and Casella, 2004;

Gamerman and Lopes, 2006) are used to estimate functionals of a distribution

function using the generated random samples. SYSTAT provides Random Sampling,

IID MC, and MCMC algorithms to generate random samples from the required target

distribution.

Random Sampling in SYSTAT enables the user to draw a number of samples, each

of a given size, from a distribution chosen from a list of 42 distributions (discrete and

continuous, univariate and multivariate) with given parameters.

If no method is known for direct generation of random samples from a given

distribution or when the density is not completely specified, then IID Monte Carlo

methods may often be suitable. The IID Monte Carlo algorithms in SYSTAT are

usable only to generate random samples from univariate continuous distributions. IID

Monte Carlo consists of two generic algorithms: Rejection Sampling and Adaptive

Rejection Sampling (ARS). In these methods an envelope (proposal) function for the

target density is used. The proposal density is such that it is feasible to draw a random

sample from it. In Rejection Sampling, the proposal distribution can be selected from

SYSTAT’s list of 20 univariate continuous distributions. In ARS, the algorithm itself

constructs an envelope (proposal) function. The ARS algorithm is applicable only for

log-concave target densities.

A Markov chain Monte Carlo (MCMC) method is used when it is possible to

generate an ergodic Markov chain whose stationary distribution is the required target

distribution. SYSTAT provides two classes of MCMC algorithms: Metropolis.

Hastings (M-H) algorithm and the Gibbs sampling algorithm. With the M-H

algorithm, random samples can be generated from univariate distributions. Three

types of the Metropolis-Hastings algorithm are available in SYSTAT: Random Walk

Metropolis-Hastings algorithm (RWM-H), Independent Metropolis-Hastings

algorithm (IndM-H), and a hybrid Metropolis-Hastings algorithm of the two. The

choice of the proposal distribution in the Metropolis-Hastings algorithms is restricted

to SYSTAT’s list of 28 univariate continuous distributions. The Gibbs Sampling

method provided is limited to the situation where full conditional univariate

distributions are defined from SYSTAT’s library of univariate distributions. It is

advisable for the user to provide a suitable initial value/distribution for the MCMC

algorithms. No convergence diagnostics are provided and it is up to the user to suggest

the burn-in period and gap in the MCMC algorithms.

From the generated random samples, estimates of means of user-given functions of

the random variable under study can be computed along with their variance estimates,

relying on the law of large numbers. A Monte Carlo Integration method can be used in

evaluating the expectation of a functional form. SYSTAT provides two Monte Carlo

Integration methods: Classical Monte Carlo integration and Importance Sampling

procedures.

IID MC and MCMC algorithms of SYSTAT generate random samples from

positive functions only. Samples generated by the Random Sampling, IID MC and

MCMC algorithms can be saved.

The user has a large role to play in the use of the IID MC and MCMC features of

SYSTAT and the success of the computations will depend largely on the user’s

judicious inputs.

Statistical Background

Drawing random samples from a given probability distribution is an important

component of any statistical Monte Carlo simulation exercise. This is usually followed

by statistical computations from the drawn samples, which can be described as Monte

Carlo integration. The random samples drawn can be used for the desired Monte Carlo

integration computations using SYSTAT. SYSTAT provides direct random sampling

facilities from a list of 42 univariate and multivariate discrete and continuous

distributions. Indeed, in statistical practice, one has to draw random samples from

several other distributions, some of which are difficult to draw directly from. The

generic IID Monte Carlo and Markov chain Monte Carlo algorithms that are provided

by SYSTAT will be of help in these contexts. The random sampling facility from the

standard distributions is a significant resource, which can be used effectively in these

generic IID and Markov chain Monte Carlo procedures.

The random sampling procedure can be used to generate random samples from the

distributions that are most commonly used for statistical work. SYSTAT implements,

as far as possible, the most efficient algorithms for generating samples from a given

type of distribution. All these depend on generation of uniform random numbers, based

on the Mersenne-Twister algorithm and Wichmann-Hill algorithm.

.. Mersenne-Twister (MT) is a pseudo random number generator developed by

Makoto Matsumoto and Takuji Nishimura (1998). Random seed for the algorithm

can be mentioned by using RSEED= seed, where seed is any integer from 1 to

4294967295 for the MT algorithm and 1 to 30000 for the Wichmann-Hill

algorithm. We recommend the MT option, especially if the number of random

numbers to be generated in your Monte Carlo studies is fairly large, say more than

10,000.

If you would like to reproduce results involving random number generation from

earlier SYSTAT versions, with old command file or otherwise, make sure that your

random number generation option (under Edit => Options => General => Random

Number Generation) is Wichmann-Hill (and, of course, that your seed is the same as

before).

The list of distributions SYSTAT generates from, expressions for associated functions,

notations used and references to their properties are given in the Volume: Data: Chapter

4: Data Transformations: Functions Relating to Probability Distributions. Definitions

of multivariate distributions, notations used and, references to their properties can be

found later in this chapter.