Markov processes for stochastic modeling [Libro electrónico] / Oliver C. Ibe

Incluye bibliografía: páginas 481-494

Markov Processes for Stochastic Modeling, 2nd Edition.. Chapter 1: Basic concepts.. Review of probability.. Random variables.. Transform methods.. Bivariate random variables.. Many random variables.. Fubini's theorem.. Sums of independent random variables.. Some probability distributions.. Introduction to stochastic processes.. Classification of stochastic processes.. Characterizing a stochastic process.. Stationary stochastic processes.. Ergodic stochastic processes.. Some models of stochastic processes.. Chapter 2: Introduction to Markov Processes.. Introduction.. Structure of markov processes.. Strong markov property.. Applications of discrete-time Markov processes.. Applications of continuous-time Markov processes.. Applications of continuous-state Markov processes.. Chapter 3: Discrete-Time Markov Chains.. Introduction.. State transition probability matrix.. State transition diagrams.. Classification of states.. Limiting-State probabilities.. Sojourn time.. Transient analysis of Discrete-Time Markov chains.. First passage and recurrence times.. Occupancy times.. Absorbing Markov chains and the fundamental matrix.. Reversible Markov chains.. Chapter 4: Continuous-Time Markov Chains.. Introduction.. Transient analysis.. Birth and death processes.. First passage time.. The Uniformization method.. Reversible Continuous-Time Markov chains.. Chapter 5: Markovian Queueing Systems.. Introduction.. Description of a queueing system.. The Kendall notation.. The Little's formula.. The PASTA property.. The M/M/1 Queueing system.. Examples of other M/M Queueing Systems.. M/G/1 Queue.. G/M/1 Queue.. Chapter 6: Markov Renewal Processes.. Renewal processes.. The renewal equation.. The elementary renewal theorem.. Random incidence and residual time.. Markov renewal process.. Semi-Markov processes.. Markov jump processes..

Chapter 7: Markovian Arrival Processes.. Introduction.. Overview of Matrix-Analytic methods.. Markovian arrival process.. Batch markovian arrival process.. Markov-Modulated Poisson process.. Markov-Modulated Bernoulli process.. Sample applications of MAP and Its derivatives.. Chapter 8: Random Walk.. Introduction.. The Two-Dimensional random walk.. Random walk as a Markov chain.. Symmetric random walk as a martingale.. Random walk with barriers.. Gambler's ruin.. First return times.. First passage times.. Maximum of a random walk.. Correlated random walk.. Continuous-time random walk.. Sample applications of random walk.. Chapter 9: Brownian Motion and Diffusion Processes.. Introduction.. Brownian motion.. Introduction to stochastic calculus.. Geometric Brownian motion.. Fractional Brownian motion.. Application of Brownian motion to option pricing.. Random walk approximation of Brownian motion.. The Ornstein-Uhlenbeck process.. Diffusion processes.. Examples of diffusion processes.. Relationship between the diffusion process and random walk.. Chapter 10: Controlled Markov Processes.. Introduction.. Markov decision processes.. Semi-Markov decision processes.. Partially observable Markov decision processes.. Chapter 11: Hidden Markov Models.. Introduction.. HMM Basics.. HMM Assumptions.. Three fundamental problems.. Solution methods.. Types of hidden Markov models.. Hidden Markov models with silent states.. Extensions of hidden Markov models.. Other Extensions of HMM.. Chapter 12: Markov Point Processes.. Point processes.. Temporal point processes.. Spatial point processes.. Spatial-Temporal point processes.. Operations on point processes.. Marked point processes.. Markov point processes.. Markov marked point processes.. Applications of Markov point processes

Disponible para usuarios de ECOSUR con su clave de acceso

Markov processes are processes that have limited memory. In particular, their dependence on the past is only through the previous state. They are used to model the behavior of many systems including communications systems, transportation networks, image segmentation and analysis, biological systems and DNA sequence analysis, random atomic motion and diffusion in physics, social mobility, population studies, epidemiology, animal and insect migration, queueing systems, resource management, dams, financial engineering, actuarial science, and decision systems. Covering a wide range of areas of application of Markov processes, this second edition is revised to highlight the most important aspects as well as the most recent trends and applications of Markov processes. The author spent over 16 years in the industry before returning to academia, and he has applied many of the principles covered in this book in multiple research projects. Therefore, this is an applications-oriented book that also includes enough theory to provide a solid ground in the subject for the reader. *Presents both the theory and applications of the different aspects of Markov processes. *Includes numerous solved examples as well as detailed diagrams that make it easier to understand the principle being presented. *Discusses different applications of hidden Markov models, such as DNA sequence analysis and speech analysis. eng

Disponible en línea

Disponible en formato PDF

Subscripción a ELSEVIER 26 de diciembre del 2013