Skip to main content

Implementing Markov Processes with Julia

Submission Number: 282
Submission ID: 4231
Submission UUID: f30ac0e2-c2b4-4303-a103-9334a5430ce9
Submission URI: /form/resource

Created: Wed, 11/29/2023 - 19:01
Completed: Wed, 11/29/2023 - 19:01
Changed: Fri, 03/14/2025 - 11:43

Remote IP address: 128.138.65.163
Submitted by: Hunter Ray
Language: English

Is draft: No
Yes
Implementing Markov Processes with Julia
Tool
Intermediate, Advanced
The following link provides an easy method of implementing Markov Decision Processes (MDP) in the Julia computing language. MDPs are a class of algorithms designed to handle stochastic situations where the actor has some level of control. For example, used at a low level, MDPs can be used to control an inverted pendulum, but applied in higher level decision making the can also decide when to take evasive action in air traffic management. MDPs can also be extended to the partially observable domain to form the Partially Observable Markov Decision Process (POMDP). This link contains a wealth of information to show one can easily implement basic POMDP and MDP algorithms and apply well known online and offline solvers.