Title: Simulation-Based Algorithms for Markov Decision Processes (Communications and Control Engineering)
Publisher: Springer
Publication Date: 2013
Language: English
ISBN 10: 144715021X
ISBN 13: 9781447150213
Binding: Hardcover
Edition: 2nd Edition
Condition: New

About this title

Synopsis

Markov decision process (MDP) models are widely used for modeling sequential decision-making problems that arise in engineering, economics, computer science, and the social sciences. Many real-world problems modeled by MDPs have huge state and/or action spaces, giving an opening to the curse of dimensionality and so making practical solution of the resulting models intractable. In other cases, the system of interest is too complex to allow explicit specification of some of the MDP model parameters, but simulation samples are readily available (e.g., for random transitions and costs). For these settings, various sampling and population-based algorithms have been developed to overcome the difficulties of computing an optimal solution in terms of a policy and/or value function. Specific approaches include adaptive sampling, evolutionary policy iteration, evolutionary random policy search, and model reference adaptive search.
This substantially enlarged new edition reflects the latest developments in novel algorithms and their underpinning theories, and presents an updated account of the topics that have emerged since the publication of the first edition. Includes:
innovative material on MDPs, both in constrained settings and with uncertain transition properties;
game-theoretic method for solving MDPs;
theories for developing roll-out based algorithms; and
details of approximation stochastic annealing, a population-based on-line simulation-based algorithm.
The self-contained approach of this book will appeal not only to researchers in MDPs, stochastic modeling, and control, and simulation but will be a valuable source of tuition and reference for students of control and operations research.

About the Author

Hyeong Soo Chang (SM’07 of the IEEE, Member of INFORMS) received the B.S. and M.S. degrees in electrical engineering and the Ph.D. degree in electrical and computer engineering, all from Purdue University,West Lafayette, IN, in 1994, 1996, and 2001, respectively. Since 2003, he has been with the Department of Computer Science and Engineering, Sogang University, Seoul, Korea, where he is now an Associate Professor. He has about 30 journal papers in the area of MDPs and related areas. His main research interests include Markov decision processes, Markov games, computational learning theory, computational intelligence, and stochastic optimization. He currently serves as an Associate Editor for the IEEE Transactions on Automatic Control.
Jiaqiao Hu (M’11 of the IEEE, Member of INFORMS) received the B.S. degree in automation from Shanghai Jiao Tong University, Shanghai, China, in 1997, the M.S. degree in applied mathematics from the University of Maryland, Baltimore County, in 2001, and the Ph.D. degree in electrical engineering from the University of Maryland, College Park, in 2006. Since 2006, he has been with the Department of Applied Mathematics and Statistics, State University of New York, Stony Brook, where he is currently an Assistant Professor Markov decision processes, simulation-based optimization, global optimization, applied probability, and stochastic modeling and analysis.
Michael Fu (Fellow of the IEEE, Member of INFORMS) received his Ph.D. and M.S degrees in applied mathematics from Harvard University in 1989 and 1986, respectively. He received S.B. and S.M. degrees in electrical engineering and an S.B. degree in mathematics from the Massachusetts Institute of Technology in 1985. Since 1989, he has been at the University of Maryland, College Park, in the College of Business and Management. He was the Simulation Area Editor for Operations and is an Associate Editor for Management Science, and has served on the Editorial Boards ofthe INFORMS Journal on Computing, Production and Operations Management and IIE Transactions. He was on the program committee for the Spring 1996 INFORMS National Meeting, in charge of contributed papers. In 1995, he received the Maryland Business School's annual Allen J. Krowe Award for Teaching Excellence. He is the co-author (with Jian-Qiang Hu) of the book, Conditional Monte Carlo: Gradient Estimation and Optimization Applications (0-7923-9873-4, 1997), which received the 1998 INFORMS College on Simulation Outstanding Publication Award. Other awards include the 1999 IIE Operations Research Division Award and a 1998 IIE Transactions Best Paper Award. In 2002, he received ISR's Outstanding Systems Engineering Faculty Award. He currently serves as a director of National Science Foundation Operations Research Program. Dr. Fu's research interests lie in the areas of stochastic derivative estimation and simulation optimization of discrete-event systems, particularly with applications towards manufacturing systems, inventory control, and the pricing of financial derivatives.
Steven I. Marcus (Fellow of the IEEE, Fellow of SIAM, Member of INFORMS) received his Ph.D. and S.M. from the Massachusetts Institute of Technology in 1975 and 1972, respectively. He received a B.A. from Rice University in 1971. From 1975 to 1991, he was with the Department of Electrical and Computer Engineering at the University of Texas at Austin, where he was the L.B. (Preach) Meaders Professor in Engineering. He was Associate Chairman of the Department during the period 1984-89. In 1991, he joined the University of Maryland, College Park, where he was Director of the Institute for Systems Research until 1996. He is currently a Professor in the Electrical Engineering Department and the Institute for Systems Research. He has served as an Editor of the SIAM Journal on Control and Optimization, and Associate Editor of Mathematics of Control, Signals, and Systems, Journal on Discrete Event Dynamic Systems, and Acta Applicandae Mathematicae. He has authored or co-authored more than 100 articles, conference proceedings, and book chapters. Dr. Marcus's research interests lie in the areas of control and systems engineering, analysis and control of stochastic systems, Markov decision processes, stochastic and adaptive control, learning, fault detection, and discrete event systems, with applications in manufacturing, acoustics, and communication networks.

"About this title" may belong to another edition of this title.

Store Description

Hello! Ria Christie Collections is an online venture that was initially set up in 2012 to sell books. We do not have a physical high street store. We are professional online booksellers. We only sell brand new books in perfect condition that we source from various suppliers and the publishers. Primarily, our aim is to provide an excellent service to all our customers. We always work as a team to achieve this. Our other objectives are to: 1. Ensure that all our products reach their destination quickly in a safe and secure manner 2. Answer to all our customer queries within 24 hours 3. Ensure that our customers are happy with their purchases 4. Provide all the items at a competitive price 5. Always listen to our customers Ria Christie Collections is not a registered company. It is a Sole Trader venture. Other key information is shown below: Contact Person Name: Rakesh Luchmun (Mr) Storefront Name: Ria Christie Collections Place of Establishment Address: Suite B; ARUN House; ARUN Building Arundel Road Uxbridge UB8 2RR United Kingdom E-Mail Address: riachristie@hotmail.co.uk VAT Number: GB 160 5650 25 We always work hard and aim to comply with all of Abebooks Policies. If you have any issues, please do not hesitate to write to us whether before or after a purchase. We promise to reply to you promptly and, in any case, within 24 hours. Thank you kindly! Yours sincerely Mr Rakesh Luchmun (Founder) and the Ria Christie Collections Team

Visit Seller's Storefront

Seller's business information

Ryefield Investments Limited
175 Pield Heath Road, Uxbridge, UB8 3NL, United Kingdom

Sale & Shipping Terms

Terms of Sale

All Returns and Refund are as per Abebooks policies.

Right of cancellation

If you are a consumer you can cancel the contract in accordance with the following. Consumer means any natural person who is acting for purposes which are outside his trade, business, craft or profession.

INFORMATION REGARDING THE RIGHT OF CANCELLATION

Statutory Right to cancel

You have the right to cancel this contract within 14 days without giving any reason.

The cancellation period will expire after 14 days from the day on which you acquire, or a third party other than the carrier and indicated by you acquires, physical possession of the the last good or the last lot or piece.

To exercise the right to cancel, you must inform us, Ria Christie Collections, Suite B; (inside) ARUN House; ARUN, Arundel Road; Uxbridge Ind Est, UB8 2RR, Uxbridge, United Kingdom, +44 7727654838, of your decision to cancel this contract by a clear statement (e.g. a letter sent by post, fax or e-mail). You may use the attached model cancellation form, but it is not obligatory. You can also electronically fill in and submit a clear statement on our website, under "My Purchases" in "My Account". If you use this option, we will communicate to you an acknowledgement of receipt of such a cancellation on a durable medium (e.g. by e-mail) without delay.

To meet the cancellation deadline, it is sufficient for you to send your communication concerning your exercise of the right to cancel before the cancellation period has expired.

Effects of cancellation

If you cancel this contract, we will reimburse to you all payments received from you, including the costs of delivery (except for the supplementary costs arising if you chose a type of delivery other than the least expensive type of standard delivery offered by us).

We may make a deduction from the reimbursement for loss in value of any goods supplied, if the loss is the result of unnecessary handling by you.

We will make the reimbursement without undue delay, and not later than 14 days after the day on which we are informed about your decision to cancel with contract.

We will make the reimbursement using the same means of payment as you used for the initial transaction, unless you have expressly agreed otherwise; in any event, you will not incur any fees as a result of such reimbursement.

We may withhold reimbursement until we have received the goods back or you have supplied evidence of having sent back the goods, whichever is the earliest.

You shall send back the goods or hand them over to us or Ria Christie Collections, Suite B; (inside) ARUN House; ARUN Building, Arundel Road; Uxbridge Ind Est, UB8 2RR, Uxbridge, United Kingdom, +44 7727654838, without undue delay and in any event not later than 14 days from the day on which you communicate your cancellation from this contract to us. The deadline is met if you send back the goods before the period of 14 days has expired. You will have to bear the direct cost of returning the goods. You are only liable for any diminished value of the goods resulting from the handling other than what is necessary to establish the nature, characteristics and functioning of the goods.

Exceptions to the right of cancellation

The right of cancellation does not apply to:

the delivery of newspapers, journals or magazines with the exception of subscription contracts; and
the supply of digital content which is not supplied on a tangible medium (e.g. on a CD or DVD) if you accepted when you placed your order that we could start to deliver it, and that you could not cancel it once delivery had started.

Model withdrawal form

(complete and return this form only if you wish to withdraw from the contract)

To: (Ria Christie Collections, Suite B; (inside) ARUN House; ARUN, Arundel Road; Uxbridge Ind Est, UB8 2RR, Uxbridge, United Kingdom, +44 7727654838)

I/We (*) hereby give notice that I/We (*) withdraw from my/our (*) contract of sale of the following goods (*)/for the provision of the following goods (*)/for the provision of the following service (*),

Ordered on (*)/received on (*)

Name of consumer(s)

Address of consumer(s)

Signature of consumer(s) (only if this form is notified on paper)

Date

* Delete as appropriate.

Shipping Terms

Orders usually ship within 2 business days. If your book order is heavy or oversized, we may contact you to let you know extra shipping is required. Thank you!

Shipping rates from United Kingdom to U.S.A.

Shipping rates from United Kingdom to U.S.A.
Order quantity	6 to 12 business days	6 to 12 business days
First item	� 11.98	� 11.98

Delivery times are set by sellers and vary by carrier and location. Orders passing through Customs may face delays and buyers are responsible for any associated duties or fees. Sellers may contact you regarding additional charges to cover any increased costs to ship your items.